diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/custom_model_fields.txt | 567 | ||||
| -rw-r--r-- | docs/flatpages.txt | 7 | ||||
| -rw-r--r-- | docs/middleware.txt | 3 | ||||
| -rw-r--r-- | docs/model-api.txt | 109 | ||||
| -rw-r--r-- | docs/serialization.txt | 27 | ||||
| -rw-r--r-- | docs/sitemaps.txt | 2 | ||||
| -rw-r--r-- | docs/templates.txt | 225 | ||||
| -rw-r--r-- | docs/templates_python.txt | 142 |
8 files changed, 945 insertions, 137 deletions
diff --git a/docs/custom_model_fields.txt b/docs/custom_model_fields.txt new file mode 100644 index 0000000000..74eb10aa82 --- /dev/null +++ b/docs/custom_model_fields.txt @@ -0,0 +1,567 @@ +=================== +Custom Model Fields +=================== + +**New in Django development version** + +Introduction +============ + +The `model reference`_ documentation explains how to use Django's standard +field classes. For many purposes, those classes are all you'll need. Sometimes, +though, the Django version won't meet your precise requirements, or you'll want +to use a field that is entirely different from those shipped with Django. + +Django's built-in field types don't cover every possible database column type -- +only the common types, such as ``VARCHAR`` and ``INTEGER``. For more obscure +column types, such as geographic polygons or even user-created types such as +`PostgreSQL custom types`_, you can define your own Django ``Field`` subclasses. + +Alternatively, you may have a complex Python object that can somehow be +serialized to fit into a standard database column type. This is another case +where a ``Field`` subclass will help you use your object with your models. + +Our example object +------------------ + +Creating custom fields requires a bit of attention to detail. To make things +easier to follow, we'll use a consistent example throughout this document. +Suppose you have a Python object representing the deal of cards in a hand of +Bridge_. It doesn't matter if you don't know how to play Bridge. You only need +to know that 52 cards are dealt out equally to four players, who are +traditionally called *north*, *east*, *south* and *west*. Our class looks +something like this:: + + class Hand(object): + def __init__(self, north, east, south, west): + # Input parameters are lists of cards ('Ah', '9s', etc) + self.north = north + self.east = east + self.south = south + self.west = west + + # ... (other possibly useful methods omitted) ... + +This is just an ordinary Python class, nothing Django-specific about it. We +would like to be able to things like this in our models (we assume the +``hand`` attribute on the model is an instance of ``Hand``):: + + + example = MyModel.objects.get(pk=1) + print example.hand.north + + new_hand = Hand(north, east, south, west) + example.hand = new_hand + example.save() + +We assign to and retrieve from the ``hand`` attribute in our model just like +any other Python class. The trick is to tell Django how to handle saving and +loading such an object. + +In order to use the ``Hand`` class in our models, we **do not** have to change +this class at all. This is ideal, because it means you can easily write +model support for existing classes where you cannot change the source code. + +.. note:: + You might only be wanting to take advantage of custom database column + types and deal with the data as standard Python types in your models; + strings, or floats, for example. This case is similar to our ``Hand`` + example and we'll note any differences as we go along. + +.. _model reference: ../model_api/ +.. _PostgreSQL custom types: http://www.postgresql.org/docs/8.2/interactive/sql-createtype.html +.. _Bridge: http://en.wikipedia.org/wiki/Contract_bridge + +Background Theory +================= + +Database storage +---------------- + +The simplest way to think of a model field is that it provides a way to take a +normal Python object -- string, boolean, ``datetime``, or something more +complex like ``Hand`` -- and convert it to and from a format that is useful +when dealing with the database (and serialization, but, as we'll see later, +that falls out fairly naturally once you have the database side under control). + +Fields in a model must somehow be converted to fit into an existing database +column type. Different databases provide different sets of valid column types, +but the rule is still the same: those are the only types you have to work +with. Anything you want to store in the database must fit into one of +those types. + +Normally, you're either writing a Django field to match a particular database +column type, or there's a fairly straightforward way to convert your data to, +say, a string. + +For our ``Hand`` example, we could convert the card data to a string of 104 +characters by concatenating all the cards together in a pre-determined order. +Say, all the *north* cards first, then the *east*, *south* and *west* cards, in +that order. So ``Hand`` objects can be saved to text or character columns in +the database. + +What does a field class do? +--------------------------- + +All of Django's fields (and when we say *fields* in this document, we always +mean model fields and not `form fields`_) are subclasses of +``django.db.models.Field``. Most of the information that Django records about a +field is common to all fields -- name, help text, validator lists, uniqueness +and so forth. Storing all that information is handled by ``Field``. We'll get +into the precise details of what ``Field`` can do later on; for now, suffice it +to say that everything descends from ``Field`` and then customises key pieces +of the class behaviour. + +.. _form fields: ../newforms/#fields + +It's important to realise that a Django field class is not what is stored in +your model attributes. The model attributes contain normal Python objects. The +field classes you define in a model are actually stored in the ``Meta`` class +when the model class is created (the precise details of how this is done are +unimportant here). This is because the field classes aren't necessary when +you're just creating and modifying attributes. Instead, they provide the +machinery for converting between the attribute value and what is stored in the +database or sent to the serializer. + +Keep this in mind when creating your own custom fields. The Django ``Field`` +subclass you write provides the machinery for converting between your Python +instances and the database/serializer values in various ways (there are +differences between storing a value and using a value for lookups, for +example). If this sounds a bit tricky, don't worry. It will hopefully become +clearer in the examples below. Just remember that you will often end up +creating two classes when you want a custom field. The first class is the +Python object that your users will manipulate. They will assign it to the model +attribute, they will read from it for displaying purposes, things like that. +This is the ``Hand`` class in our example. The second class is the ``Field`` +subclass. This is the class that knows how to convert your first class back and +forth between its permanent storage form and the Python form. + +Writing a ``Field`` subclass +============================= + +When you are planning your ``Field`` subclass, first give some thought to +which existing field your new field is most similar to. Can you subclass an +existing Django field and save yourself some work? If not, you should subclass the ``Field`` class, from which everything is descended. + +Initialising your new field is a matter of separating out any arguments that +are specific to your case from the common arguments and passing the latter to +the ``__init__()`` method of ``Field`` (or your parent class). + +In our example, the Django field we create is going to be called +``HandField``. It's not a bad idea to use a similar naming scheme to Django's +fields so that our new class is identifiable and yet clearly related to the +``Hand`` class it is wrapping. It doesn't behave like any existing field, so +we'll subclass directly from ``Field``:: + + from django.db import models + + class HandField(models.Field): + def __init__(self, *args, **kwargs): + kwargs['max_length'] = 104 + super(HandField, self).__init__(*args, **kwargs) + +Our ``HandField`` will accept most of the standard field options (see the list +below), but we ensure it has a fixed length, since it only needs to hold 52 +card values plus their suits; 104 characters in total. + +.. note:: + Many of Django's model fields accept options that they don't do anything + with. For example, you can pass both ``editable`` and ``auto_now`` to a + ``DateField`` and it will simply ignore the ``editable`` parameter + (``auto_now`` being set implies ``editable=False``). No error is raised in + this case. + + This behaviour simplifies the field classes, because they don't need to + check for options that aren't necessary. They just pass all the options to + the parent class and then don't use them later on. It is up to you whether + you want your fields to be more strict about the options they select, or + to use the simpler, more permissive behaviour of the current fields. + +The ``Field.__init__()`` method takes the following parameters, in this +order: + + - ``verbose_name`` + - ``name`` + - ``primary_key`` + - ``max_length`` + - ``unique`` + - ``blank`` + - ``null`` + - ``db_index`` + - ``core`` + - ``rel``: Used for related fields (like ``ForeignKey``). For advanced use + only. + - ``default`` + - ``editable`` + - ``serialize``: If ``False``, the field will not be serialized when the + model is passed to Django's serializers_. Defaults to ``True``. + - ``prepopulate_from`` + - ``unique_for_date`` + - ``unique_for_month`` + - ``unique_for_year`` + - ``validator_list`` + - ``choices`` + - ``radio_admin`` + - ``help_text`` + - ``db_column`` + - ``db_tablespace``: Currently only used with the Oracle backend and only + for index creation. You can usually ignore this option. + +All of the options without an explanation in the above list have the same +meaning they do for normal Django fields. See the `model documentation`_ for +examples and details. + +.. _serializers: ../serialization/ +.. _model documentation: ../model-api/ + +The ``SubfieldBase`` metaclass +------------------------------ + +As we indicated in the introduction_, field subclasses are often needed for +two reasons. Either to take advantage of a custom database column type, or to +handle complex Python types. A combination of the two is obviously also +possible. If you are only working with custom database column types and your +model fields appear in Python as standard Python types direct from the +database backend, you don't need to worry about this section. + +If you are handling custom Python types, such as our ``Hand`` class, we need +to make sure that when Django initialises an instance of our model and assigns +a database value to our custom field attribute we convert that value into the +appropriate Python object. The details of how this happens internally are a +little complex. For the field writer, though, things are fairly simple. Make +sure your field subclass uses ``django.db.models.SubfieldBase`` as its +metaclass. This ensures that the ``to_python()`` method, documented below_, +will always be called when the attribute is initialised. + +Our ``HandField`` class now looks like this:: + + class HandField(models.Field): + __metaclass__ = models.SubfieldBase + + def __init__(self, *args, **kwargs): + # ... + +.. _below: #to-python-self-value + +Useful methods +-------------- + +Once you've created your ``Field`` subclass and setup up the +``__metaclass__``, if necessary, there are a few standard methods you need to +consider overriding. Which of these you need to implement will depend on you +particular field behaviour. The list below is in approximately decreasing +order of importance, so start from the top. + +``db_type(self)`` +~~~~~~~~~~~~~~~~~ + +Returns the database column data type for the ``Field``, taking into account +the current ``DATABASE_ENGINE`` setting. + +Say you've created a PostgreSQL custom type called ``mytype``. You can use this +field with Django by subclassing ``Field`` and implementing the ``db_type()`` +method, like so:: + + from django.db import models + + class MytypeField(models.Field): + def db_type(self): + return 'mytype' + +Once you have ``MytypeField``, you can use it in any model, just like any other +``Field`` type:: + + class Person(models.Model): + name = models.CharField(max_length=80) + gender = models.CharField(max_length=1) + something_else = MytypeField() + +If you aim to build a database-agnostic application, you should account for +differences in database column types. For example, the date/time column type +in PostgreSQL is called ``timestamp``, while the same column in MySQL is called +``datetime``. The simplest way to handle this in a ``db_type()`` method is to +import the Django settings module and check the ``DATABASE_ENGINE`` setting. +For example:: + + class MyDateField(models.Field): + def db_type(self): + from django.conf import settings + if settings.DATABASE_ENGINE == 'mysql': + return 'datetime' + else: + return 'timestamp' + +The ``db_type()`` method is only called by Django when the framework constructs +the ``CREATE TABLE`` statements for your application -- that is, when you first +create your tables. It's not called at any other time, so it can afford to +execute slightly complex code, such as the ``DATABASE_ENGINE`` check in the +above example. + +Some database column types accept parameters, such as ``CHAR(25)``, where the +parameter ``25`` represents the maximum column length. In cases like these, +it's more flexible if the parameter is specified in the model rather than being +hard-coded in the ``db_type()`` method. For example, it wouldn't make much +sense to have a ``CharMaxlength25Field``, shown here:: + + # This is a silly example of hard-coded parameters. + class CharMaxlength25Field(models.Field): + def db_type(self): + return 'char(25)' + + # In the model: + class MyModel(models.Model): + # ... + my_field = CharMaxlength25Field() + +The better way of doing this would be to make the parameter specifiable at run +time -- i.e., when the class is instantiated. To do that, just implement +``__init__()``, like so:: + + # This is a much more flexible example. + class BetterCharField(models.Field): + def __init__(self, max_length, *args, **kwargs): + self.max_length = max_length + super(BetterCharField, self).__init__(*args, **kwargs) + + def db_type(self): + return 'char(%s)' % self.max_length + + # In the model: + class MyModel(models.Model): + # ... + my_field = BetterCharField(25) + +Finally, if your column requires truly complex SQL setup, return ``None`` from +``db_type()``. This will cause Django's SQL creation code to skip over this +field. You are then responsible for creating the column in the right table in +some other way, of course, but this gives you a way to tell Django to get out +of the way. + + +``to_python(self, value)`` +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Converts between all the ways your field can receive its initial value and the +Python object you want to end up with. The default version just returns +``value``, so is useful is the database backend returns the data already in +the correct form (a Python string, for example). + +Normally, you will need to override this method. As a general rule, be +prepared to accept an instance of the right type (e.g. ``Hand`` in our ongoing +example), a string (from a deserializer, for example), and whatever the +database wrapper returns for the column type you are using. + +In our ``HandField`` class, we are storing the data in a character field in +the database, so we need to be able to process strings and ``Hand`` instances +in ``to_python()``:: + + class HandField(models.Field): + # ... + + def to_python(self, value): + if isinstance(value, Hand): + return value + + # The string case + p1 = re.compile('.{26}') + p2 = re.compile('..') + args = [p2.findall(x) for x in p1.findall(value)] + return Hand(*args) + +Notice that we always return a ``Hand`` instance from this method. That is the +Python object we want to store in the model's attribute. + +``get_db_prep_save(self, value)`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This is the reverse of ``to_python()`` when working with the database backends +(as opposed to serialization). The ``value`` parameter is the current value of +the model's attribute (a field has no reference to its containing model, so it +cannot retrieve the value itself) and the method should return data in a +format that can be used as a parameter in a query for the database backend. + +For example:: + + class HandField(models.Field): + # ... + + def get_db_prep_save(self, value): + return ''.join([''.join(l) for l in (self.north, + self.east, self.south, self.west)]) + + +``pre_save(self, model_instance, add)`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This method is called just prior to ``get_db_prep_save()`` and should return +the value of the appropriate attribute from ``model_instance`` for this field. +The attribute name is in ``self.attname`` (this is set up by ``Field``). If +the model is being saved to the database for the first time, the ``add`` +parameter will be ``True``, otherwise it will be ``False``. + +Often you won't need to override this method. However, at times it can be very +useful. For example, the Django ``DateTimeField`` uses this method to set the +attribute to the correct value before returning it in the cases when +``auto_now`` or ``auto_now_add`` are set on the field. + +If you do override this method, you must return the value of the attribute at +the end. You should also update the model's attribute if you make any changes +to the value so that code holding references to the model will always see the +correct value. + +``get_db_prep_lookup(self, lookup_type, value)`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Prepares the ``value`` for passing to the database when used in a lookup (a +``WHERE`` constraint in SQL). The ``lookup_type`` will be one of the valid +Django filter lookups: ``exact``, ``iexact``, ``contains``, ``icontains``, +``gt``, ``gte``, ``lt``, ``lte``, ``in``, ``startswith``, ``istartswith``, +``endswith``, ``iendswith``, ``range``, ``year``, ``month``, ``day``, +``isnull``, ``search``, ``regex``, and ``iregex``. + +Your method must be prepared to handle all of these ``lookup_type`` values and +should raise either a ``ValueError`` if the ``value`` is of the wrong sort (a +list when you were expecting an object, for example) or a ``TypeError`` if +your field does not support that type of lookup. For many fields, you can get +by with handling the lookup types that need special handling for your field +and pass the rest of the ``get_db_prep_lookup()`` method of the parent class. + +If you needed to implement ``get_db_prep_save()``, you will usually need to +implement ``get_db_prep_lookup()``. The usual reason is because of the +``range`` and ``in`` lookups. In these case, you will passed a list of +objects (presumably of the right type) and will need to convert them to a list +of things of the right type for passing to the database. Sometimes you can +reuse ``get_db_prep_save()``, or at least factor out some common pieces from +both methods into a help function. + +For example:: + + class HandField(models.Field): + # ... + + def get_db_prep_lookup(self, lookup_type, value): + # We only handle 'exact' and 'in'. All others are errors. + if lookup_type == 'exact': + return self.get_db_prep_save(value) + elif lookup_type == 'in': + return [self.get_db_prep_save(v) for v in value] + else: + raise TypeError('Lookup type %r not supported.' % lookup_type) + + +``formfield(self, form_class=forms.CharField, **kwargs)`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Returns the default form field to use when this field is displayed +in a model. This method is called by the `helper functions`_ +``form_for_model()`` and ``form_for_instance()``. + +All of the ``kwargs`` dictionary is passed directly to the form field's +``__init__()`` method. Normally, all you need to do is set up a good default +for the ``form_class`` argument and then delegate further handling to the +parent class. This might require you to write a custom form field (and even a +form widget). See the `forms documentation`_ for information about this. Also +have a look at ``django.contrib.localflavor`` for some examples of custom +widgets. + +Continuing our ongoing example, we can write the ``formfield()`` method as:: + + class HandField(models.Field): + # ... + + def formfield(self, **kwargs): + # This is a fairly standard way to set up some defaults + # whilst letting the caller override them. + defaults = {'form_class': MyFormField} + defaults.update(kwargs) + return super(HandField, self).formfield(**defaults) + +This assumes we have some ``MyFormField`` field class (which has its own +default widget) imported. This document doesn't cover the details of writing +custom form fields. + +.. _helper functions: ../newforms/#generating-forms-for-models +.. _forms documentation: ../newforms/ + +``get_internal_type(self)`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Returns a string giving the name of the ``Field`` subclass we are emulating at +the database level. This is used to determine the type of database column for +simple cases. + +If you have created a ``db_type()`` method, you do not need to worry about +``get_internal_type()`` -- it won't be used much. Sometimes, though, your +database storage is similar in type to some other field, so you can use that +other field's logic to create the right column. + +For example:: + + class HandField(models.Field): + # ... + + def get_internal_type(self): + return 'CharField' + +No matter which database backend we are using, this will mean that ``syncdb`` +and other SQL commands create the right column type for storing a string. + +If ``get_internal_type()`` returns a string that is not known to Django for +the database backend you are using -- that is, it doesn't appear in +``django.db.backends.<db_name>.creation.DATA_TYPES`` -- the string will still +be used by the serializer, but the default ``db_type()`` method will return +``None``. See the documentation of ``db_type()`` above_ for reasons why this +might be useful. Putting a descriptive string in as the type of the field for +the serializer is a useful idea if you are ever going to be using the +serializer output in some other place, outside of Django. + +.. _above: #db-type-self + +``flatten_data(self, follow, obj=None)`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. admonition:: Subject to change + + Although implementing this method is necessary to allow field + serialization, the API might change in the future. + +Returns a dictionary, mapping the field's attribute name to a flattened string +version of the data. This method has some internal uses that aren't of +interest to use here (mostly having to do with manipulators). For our +purposes, it is sufficient to return a one item dictionary that maps the +attribute name to a string. + +This method is used by the serializers to convert the field into a string for +output. You can ignore the input parameters for serialization purposes, +although calling ``Field._get_val_from_obj(obj)`` is the best way to get the +value to serialize. + +For example, since our ``HandField`` uses strings for its data storage anyway, +we can reuse some existing conversion code:: + + class HandField(models.Field): + # ... + + def flatten_data(self, follow, obj=None): + value = self._get_val_from_obj(obj) + return {self.attname: self.get_db_prep_save(value)} + +Some general advice +-------------------- + +Writing a custom field can be a tricky process sometimes, particularly if you +are doing complex conversions between your Python types and your database and +serialization formats. A couple of tips to make things go more smoothly: + + 1. Look at the existing Django fields (in + ``django/db/models/fields/__init__.py``) for inspiration. Try to find a field + that is already close to what you want and extend it a little bit, in + preference to creating an entirely new field from scratch. + + 2. Put a ``__str__()`` or ``__unicode__()`` method on the class you are + wrapping up as a field. There are a lot of places where the default behaviour + of the field code is to call ``force_unicode()`` on the value (in our + examples in this document, ``value`` would be a ``Hand`` instance, not a + ``HandField``). So if your ``__unicode__()`` method automatically converts to + the string form of your Python object, you can save yourself a lot of work. + diff --git a/docs/flatpages.txt b/docs/flatpages.txt index d082090689..7c27fe8793 100644 --- a/docs/flatpages.txt +++ b/docs/flatpages.txt @@ -113,3 +113,10 @@ Here's a sample ``flatpages/default.html`` template:: {{ flatpage.content }} </body> </html> + +Since you're already entering raw HTML into the admin page for a flatpage, +both ``flatpage.title`` and ``flatpage.content`` are marked as **not** +requiring `automatic HTML escaping`_ in the template. + +.. _automatic HTML escaping: ../templates/#automatic-html-escaping + diff --git a/docs/middleware.txt b/docs/middleware.txt index 30a4899a3e..41b1a96b89 100644 --- a/docs/middleware.txt +++ b/docs/middleware.txt @@ -104,8 +104,7 @@ Handles conditional GET operations. If the response has a ``ETag`` or ``Last-Modified`` header, and the request has ``If-None-Match`` or ``If-Modified-Since``, the response is replaced by an HttpNotModified. -Also removes the content from any response to a HEAD request and sets the -``Date`` and ``Content-Length`` response-headers. +Also sets the ``Date`` and ``Content-Length`` response-headers. django.middleware.http.SetRemoteAddrFromForwardedFor ---------------------------------------------------- diff --git a/docs/model-api.txt b/docs/model-api.txt index b49963d8f5..ca84c84d09 100644 --- a/docs/model-api.txt +++ b/docs/model-api.txt @@ -1013,111 +1013,12 @@ See the `One-to-one relationship model example`_ for a full example. Custom field types ------------------ -**New in Django development version** - -Django's built-in field types don't cover every possible database column type -- -only the common types, such as ``VARCHAR`` and ``INTEGER``. For more obscure -column types, such as geographic polygons or even user-created types such as -`PostgreSQL custom types`_, you can define your own Django ``Field`` subclasses. - -.. _PostgreSQL custom types: http://www.postgresql.org/docs/8.2/interactive/sql-createtype.html - -.. admonition:: Experimental territory - - This is an area of Django that traditionally has not been documented, but - we're starting to include bits of documentation, one feature at a time. - Please forgive the sparseness of this section. - - If you like living on the edge and are comfortable with the risk of - unstable, undocumented APIs, see the code for the core ``Field`` class - in ``django/db/models/fields/__init__.py`` -- but if/when the innards - change, don't say we didn't warn you. - -To create a custom field type, simply subclass ``django.db.models.Field``. -Here is an incomplete list of the methods you should implement: - -``db_type()`` -~~~~~~~~~~~~~ - -Returns the database column data type for the ``Field``, taking into account -the current ``DATABASE_ENGINE`` setting. - -Say you've created a PostgreSQL custom type called ``mytype``. You can use this -field with Django by subclassing ``Field`` and implementing the ``db_type()`` -method, like so:: - - from django.db import models - - class MytypeField(models.Field): - def db_type(self): - return 'mytype' - -Once you have ``MytypeField``, you can use it in any model, just like any other -``Field`` type:: - - class Person(models.Model): - name = models.CharField(max_length=80) - gender = models.CharField(max_length=1) - something_else = MytypeField() - -If you aim to build a database-agnostic application, you should account for -differences in database column types. For example, the date/time column type -in PostgreSQL is called ``timestamp``, while the same column in MySQL is called -``datetime``. The simplest way to handle this in a ``db_type()`` method is to -import the Django settings module and check the ``DATABASE_ENGINE`` setting. -For example:: - - class MyDateField(models.Field): - def db_type(self): - from django.conf import settings - if settings.DATABASE_ENGINE == 'mysql': - return 'datetime' - else: - return 'timestamp' - -The ``db_type()`` method is only called by Django when the framework constructs -the ``CREATE TABLE`` statements for your application -- that is, when you first -create your tables. It's not called at any other time, so it can afford to -execute slightly complex code, such as the ``DATABASE_ENGINE`` check in the -above example. - -Some database column types accept parameters, such as ``CHAR(25)``, where the -parameter ``25`` represents the maximum column length. In cases like these, -it's more flexible if the parameter is specified in the model rather than being -hard-coded in the ``db_type()`` method. For example, it wouldn't make much -sense to have a ``CharMaxlength25Field``, shown here:: - - # This is a silly example of hard-coded parameters. - class CharMaxlength25Field(models.Field): - def db_type(self): - return 'char(25)' - - # In the model: - class MyModel(models.Model): - # ... - my_field = CharMaxlength25Field() - -The better way of doing this would be to make the parameter specifiable at run -time -- i.e., when the class is instantiated. To do that, just implement -``__init__()``, like so:: - - # This is a much more flexible example. - class BetterCharField(models.Field): - def __init__(self, max_length, *args, **kwargs): - self.max_length = max_length - super(BetterCharField, self).__init__(*args, **kwargs) - - def db_type(self): - return 'char(%s)' % self.max_length - - # In the model: - class MyModel(models.Model): - # ... - my_field = BetterCharField(25) +If one of the existing model fields cannot be used to fit your purposes, or if +you wish to take advantage of some less common database column types, you can +create your own field class. Full coverage of creating your own fields is +provided in the `Custom Model Fields`_ documentation. -Note that if you implement ``__init__()`` on a ``Field`` subclass, it's -important to call ``Field.__init__()`` -- i.e., the parent class' -``__init__()`` method. +.. _Custom Model Fields: ../custom_model_fields/ Meta options ============ diff --git a/docs/serialization.txt b/docs/serialization.txt index fa9b4edd51..cf8b196931 100644 --- a/docs/serialization.txt +++ b/docs/serialization.txt @@ -47,14 +47,14 @@ This is useful if you want to serialize data directly to a file-like object Subset of fields ~~~~~~~~~~~~~~~~ -If you only want a subset of fields to be serialized, you can +If you only want a subset of fields to be serialized, you can specify a ``fields`` argument to the serializer:: from django.core import serializers data = serializers.serialize('xml', SomeModel.objects.all(), fields=('name','size')) In this example, only the ``name`` and ``size`` attributes of each model will -be serialized. +be serialized. .. note:: @@ -111,9 +111,9 @@ Django "ships" with a few included serializers: ``python`` Translates to and from "simple" Python objects (lists, dicts, strings, etc.). Not really all that useful on its own, but used as a base for other serializers. - - ``yaml`` Serializes to YAML (Yet Another Markup Lanuage). This - serializer is only available if PyYAML_ is installed. + + ``yaml`` Serializes to YAML (Yet Another Markup Lanuage). This + serializer is only available if PyYAML_ is installed. ========== ============================================================== .. _json: http://json.org/ @@ -135,6 +135,23 @@ For example:: json_serializer = serializers.get_serializer("json")() json_serializer.serialize(queryset, ensure_ascii=False, stream=response) +Django ships with a copy of simplejson_ in the source. Be aware, that if +you're using that for serializing directly that not all Django output can be +passed unmodified to simplejson. In particular, `lazy translation objects`_ +need a `special encoder`_ written for them. Something like this will work:: + + from django.utils.functional import Promise + from django.utils.encoding import force_unicode + + class LazyEncoder(simplejson.JSONEncoder): + def default(self, obj): + if isinstance(obj, Promise): + return force_unicode(obj) + return obj + +.. _lazy translation objects: ../i18n/#lazy-translation +.. _special encoder: http://svn.red-bean.com/bob/simplejson/tags/simplejson-1.7/docs/index.html + Writing custom serializers `````````````````````````` diff --git a/docs/sitemaps.txt b/docs/sitemaps.txt index 1d4fba2626..eb749dda2f 100644 --- a/docs/sitemaps.txt +++ b/docs/sitemaps.txt @@ -47,7 +47,7 @@ Initialization ============== To activate sitemap generation on your Django site, add this line to your -URLconf_: +URLconf_:: (r'^sitemap.xml$', 'django.contrib.sitemaps.views.sitemap', {'sitemaps': sitemaps}) diff --git a/docs/templates.txt b/docs/templates.txt index 5d5f657747..07258bb46a 100644 --- a/docs/templates.txt +++ b/docs/templates.txt @@ -280,7 +280,9 @@ Here are some tips for working with inheritance: * If you need to get the content of the block from the parent template, the ``{{ block.super }}`` variable will do the trick. This is useful if you want to add to the contents of a parent block instead of - completely overriding it. + completely overriding it. Data inserted using ``{{ block.super }}`` will + not be automatically escaped (see the `next section`_), since it was + already escaped, if necessary, in the parent template. * For extra readability, you can optionally give a *name* to your ``{% endblock %}`` tag. For example:: @@ -299,6 +301,132 @@ it also defines the content that fills the hole in the *parent*. If there were two similarly-named ``{% block %}`` tags in a template, that template's parent wouldn't know which one of the blocks' content to use. +.. _next section: #automatic-html-escaping + +Automatic HTML escaping +======================= + +**New in Django development version** + +A very real problem when creating HTML (and other) output using templates and +variable substitution is the possibility of accidently inserting some variable +value that affects the resulting HTML. For example, a template fragment such as +:: + + Hello, {{ name }}. + +seems like a harmless way to display the user's name. However, if you are +displaying data that the user entered directly and they had entered their name as :: + + <script>alert('hello')</script> + +this would always display a Javascript alert box when the page was loaded. +Similarly, if you were displaying some data generated by another process and it +contained a '<' symbol, you couldn't just dump this straight into your HTML, +because it would be treated as the start of an element. The effects of these +sorts of problems can vary from merely annoying to allowing exploits via `Cross +Site Scripting`_ (XSS) attacks. + +.. _Cross Site Scripting: http://en.wikipedia.org/wiki/Cross-site_scripting + +In order to provide some protection against these problems, Django +provides automatic (but controllable) HTML escaping for data coming from +tempate variables. Inside this tag, any data that comes from template +variables is examined to see if it contains one of the five HTML characters +(<, >, ', " and &) that often need escaping and those characters are converted +to their respective HTML entities. It causes no harm if a character is +converted to an entity when it doesn't need to be, so all five characters are +always converted. + +Since some variables will contain data that is *intended* to be rendered +as HTML, template tag and filter writers can mark their output strings as +requiring no further escaping. For example, the ``unordered_list`` filter is +designed to return raw HTML and we want the template processor to simply +display the results as returned, without applying any escaping. That is taken +care of by the filter. The template author need do nothing special in that +case. + +By default, automatic HTML escaping is always applied. However, sometimes you +will not want this to occur (for example, if you're using the templating +system to create an email). To control automatic escaping inside your template, +wrap the affected content in the ``autoescape`` tag, like so:: + + {% autoescape off %} + Hello {{ name }} + {% endautoescape %} + +The auto-escaping tag passes its effect onto templates that extend the +current one as well as templates included via the ``include`` tag, just like +all block tags. + +The ``autoescape`` tag takes either ``on`` or ``off`` as its argument. At times, you might want to force auto-escaping when it would otherwise be disabled. For example:: + + Auto-escaping is on by default. Hello {{ name }} + + {% autoescape off %} + This will not be auto-escaped: {{ data }}. + + Nor this: {{ other_data }} + {% autoescape on %} + Auto-escaping applies again, {{ name }} + {% endautoescape %} + {% endautoescape %} + +For individual variables, the ``safe`` filter can also be used to indicate +that the contents should not be automatically escaped:: + + This will be escaped: {{ data }} + This will not be escaped: {{ data|safe }} + +Think of *safe* as shorthand for *safe from further escaping* or *can be +safely interpreted as HTML*. In this example, if ``data`` contains ``'<a>'``, +the output will be:: + + This will be escaped: <a> + This will not be escaped: <a> + +Generally, you won't need to worry about auto-escaping very much. View +developers and custom filter authors need to think about when their data +shouldn't be escaped and mark it appropriately. They are in a better position +to know when that should happen than the template author, so it is their +responsibility. By default, all output is escaped unless the template +processor is explicitly told otherwise. + +You should also note that if you are trying to write a template that might be +used in situations where automatic escaping is enabled or disabled and you +don't know which (such as when your template is included in other templates), +you can safely write as if you were in an ``{% autoescape off %}`` situation. +Scatter ``escape`` filters around for any variables that need escaping. When +auto-escaping is on, these extra filters won't change the output -- any +variables that use the ``escape`` filter do not have further automatic +escaping applied to them. + +String literals and automatic escaping +-------------------------------------- + +Sometimes you will pass a string literal as an argument to a filter. For +example:: + + {{ data|default:"This is a string literal." }} + +All string literals are inserted **without** any automatic escaping into the +template, if they are used (it's as if they were all passed through the +``safe`` filter). The reasoning behind this is that the template author is in +control of what goes into the string literal, so they can make sure the text +is correctly escaped when the template is written. + +This means you would write :: + + {{ data|default:"3 > 2" }} + +...rather than :: + + {{ data|default:"3 > 2" }} <-- Bad! Don't do this. + +This doesn't affect what happens to data coming from the variable itself. +The variable's contents are still automatically escaped, if necessary, since +they're beyond the control of the template author. + Using the built-in reference ============================ @@ -374,6 +502,24 @@ available, and what they do. Built-in tag reference ---------------------- +autoescape +~~~~~~~~~~ + +**New in Django development version** + +Control the current auto-escaping behaviour. This tag takes either ``on`` or +``off`` as an argument and that determines whether auto-escaping is in effect +inside the block. + +When auto-escaping is in effect, all variable content has HTML escaping applied +to it before placing the result into the output (but after any filters have +been applied). This is equivalent to manually applying the ``escape`` filter +attached to each variable. + +The only exceptions are variables that are already marked as 'safe' from +escaping, either by the code that populated the variable, or because it has +the ``safe`` or ``escape`` filters applied. + block ~~~~~ @@ -452,7 +598,7 @@ just like in variable syntax. Sample usage:: - {% filter escape|lower %} + {% filter force_escape|lower %} This text will be HTML-escaped, and will appear in all lowercase. {% endfilter %} @@ -740,7 +886,7 @@ Available format strings: if they're zero and the special-case strings 'midnight' and 'noon' if appropriate. Proprietary extension. - r RFC 822 formatted date. ``'Thu, 21 Dec 2000 16:01:07 +0200'`` + r RFC 2822 formatted date. ``'Thu, 21 Dec 2000 16:01:07 +0200'`` s Seconds, 2 digits with leading zeros. ``'00'`` to ``'59'`` S English ordinal suffix for day of the ``'st'``, ``'nd'``, ``'rd'`` or ``'th'`` month, 2 characters. @@ -1076,6 +1222,10 @@ Returns true if the value is divisible by the argument. escape ~~~~~~ +**New in Django development version:** The behaviour of this filter has +changed slightly in the development version (the affects are only applied +once, after all other filters). + Escapes a string's HTML. Specifically, it makes these replacements: * ``"&"`` to ``"&"`` @@ -1084,6 +1234,16 @@ Escapes a string's HTML. Specifically, it makes these replacements: * ``'"'`` (double quote) to ``'"'`` * ``"'"`` (single quote) to ``'''`` +The escaping is only applied when the string is output, so it does not matter +where in a chained sequence of filters you put ``escape``: it will always be +applied as though it were the last filter. If you want escaping to be applied +immediately, use the ``force_escape`` filter. + +Applying ``escape`` to a variable that would normally have auto-escaping +applied to the result will only result in one round of escaping being done. So +it is safe to use this function even in auto-escaping environments. If you want +multiple escaping passes to be applied, use the ``force_escape`` filter. + filesizeformat ~~~~~~~~~~~~~~ @@ -1106,25 +1266,50 @@ floatformat When used without an argument, rounds a floating-point number to one decimal place -- but only if there's a decimal part to be displayed. For example: - * ``36.123`` gets converted to ``36.1`` - * ``36.15`` gets converted to ``36.2`` - * ``36`` gets converted to ``36`` +======== ======================= ====== +value Template Output +======== ======================= ====== +34.23234 {{ value|floatformat }} 34.2 +34.00000 {{ value|floatformat }} 34 +34.26000 {{ value|floatformat }} 34.3 +======== ======================= ====== + +If used with a numeric integer argument, ``floatformat`` rounds a number to +that many decimal places. For example: -If used with a numeric integer argument, ``floatformat`` rounds a number to that -many decimal places. For example: +======== ========================= ====== +value Template Output +======== ========================= ====== +34.23234 {{ value|floatformat:3 }} 34.232 +34.00000 {{ value|floatformat:3 }} 34.000 +34.26000 {{ value|floatformat:3 }} 34.260 +======== ========================= ====== - * ``36.1234`` with floatformat:3 gets converted to ``36.123`` - * ``36`` with floatformat:4 gets converted to ``36.0000`` +If the argument passed to ``floatformat`` is negative, it will round a number +to that many decimal places -- but only if there's a decimal part to be +displayed. For example: -If the argument passed to ``floatformat`` is negative, it will round a number to -that many decimal places -- but only if there's a decimal part to be displayed. -For example: +======== ============================ ====== +value Template Output +======== ============================ ====== +34.23234 {{ value|floatformat:"-3" }} 34.232 +34.00000 {{ value|floatformat:"-3" }} 34 +34.26000 {{ value|floatformat:"-3" }} 34.260 +======== ============================ ====== - * ``36.1234`` with floatformat:-3 gets converted to ``36.123`` - * ``36`` with floatformat:-4 gets converted to ``36`` +Using ``floatformat`` with no argument is equivalent to using ``floatformat`` +with an argument of ``-1``. -Using ``floatformat`` with no argument is equivalent to using ``floatformat`` with -an argument of ``-1``. +force_escape +~~~~~~~~~~~~ + +**New in Django development version** + +Applies HTML escaping to a string (see the ``escape`` filter for details). +This filter is applied *immediately* and returns a new, escaped string. This +is useful in the rare cases where you need multiple escaping or want to apply +other filters to the escaped results. Normally, you want to use the ``escape`` +filter. get_digit ~~~~~~~~~ @@ -1250,6 +1435,12 @@ Right-aligns the value in a field of a given width. **Argument:** field size +safe +~~~~ + +Marks a string as not requiring further HTML escaping prior to output. When +autoescaping is off, this filter has no effect. + slice ~~~~~ diff --git a/docs/templates_python.txt b/docs/templates_python.txt index bd105888ce..e4658f6461 100644 --- a/docs/templates_python.txt +++ b/docs/templates_python.txt @@ -219,13 +219,13 @@ be replaced with the name of the invalid variable. While ``TEMPLATE_STRING_IF_INVALID`` can be a useful debugging tool, it is a bad idea to turn it on as a 'development default'. - + Many templates, including those in the Admin site, rely upon the silence of the template system when a non-existent variable is encountered. If you assign a value other than ``''`` to ``TEMPLATE_STRING_IF_INVALID``, you will experience rendering problems with these templates and sites. - + Generally, ``TEMPLATE_STRING_IF_INVALID`` should only be enabled in order to debug a specific template problem, then cleared once debugging is complete. @@ -722,6 +722,95 @@ decorator instead:: If you leave off the ``name`` argument, as in the second example above, Django will use the function's name as the filter name. +Filters and auto-escaping +~~~~~~~~~~~~~~~~~~~~~~~~~ + +**New in Django development version** + +When you are writing a custom filter, you need to give some thought to how +this filter will interact with Django's auto-escaping behaviour. Firstly, you +should realise that there are three types of strings that can be passed around +inside the template code: + + * raw strings are the native Python ``str`` or ``unicode`` types. On + output, they are escaped if auto-escaping is in effect and presented + unchanged, otherwise. + + * "safe" strings are strings that are safe from further escaping at output + time. Any necessary escaping has already been done. They are commonly used + for output that contains raw HTML that is intended to be intrepreted on the + client side. + + Internally, these strings are of type ``SafeString`` or ``SafeUnicode``, + although they share a common base class in ``SafeData``, so you can test + for them using code like:: + + if isinstance(value, SafeData): + # Do something with the "safe" string. + + * strings which are marked as "needing escaping" are *always* escaped on + output, regardless of whether they are in an ``autoescape`` block or not. + These strings are only escaped once, however, even if auto-escaping + applies. This type of string is internally represented by the types + ``EscapeString`` and ``EscapeUnicode``. You will not normally need to worry + about these; they exist for the implementation of the ``escape`` filter. + +Inside your filter, you will need to think about three areas in order to be +auto-escaping compliant: + + 1. If your filter returns a string that is ready for direct output (it should + be considered a "safe" string), you should call + ``django.utils.safestring.mark_safe()`` on the result prior to returning. + This will turn the result into the appropriate ``SafeData`` type. This is + often the case when you are returning raw HTML, for example. + + 2. If your filter is given a "safe" string, is it guaranteed to return a + "safe" string? If so, set the ``is_safe`` attribute on the function to be + ``True``. For example, a filter that replaced a word consisting only of + digits with the number spelt out in words is going to be + safe-string-preserving, since it cannot introduce any of the five dangerous + characters: <, >, ", ' or &. We can write:: + + @register.filter + def convert_to_words(value): + # ... implementation here ... + return result + + convert_to_words.is_safe = True + + Note that this filter does not return a universally safe result (it does + not return ``mark_safe(result)``) because if it is handed a raw string such + as '<a>', this will need further escaping in an auto-escape environment. + The ``is_safe`` attribute only talks about the the result when a safe + string is passed into the filter. + + 3. Will your filter behave differently depending upon whether auto-escaping + is currently in effect or not? This is normally a concern when you are + returning mixed content (HTML elements mixed with user-supplied content). + For example, the ``ordered_list`` filter that ships with Django needs to + know whether to escape its content or not. It will always return a safe + string. Since it returns raw HTML, we cannot apply escaping to the + result -- it needs to be done in-situ. + + For these cases, the filter function needs to be told what the current + auto-escaping setting is. Set the ``needs_autoescape`` attribute on the + filter to ``True`` and have your function take an extra argument called + ``autoescape`` with a default value of ``None``. When the filter is called, + the ``autoescape`` keyword argument will be ``True`` if auto-escaping is in + effect. For example, the ``unordered_list`` filter is written as:: + + def unordered_list(value, autoescape=None): + # ... lots of code here ... + + return mark_safe(...) + + unordered_list.is_safe = True + unordered_list.needs_autoescape = True + +By default, both the ``is_safe`` and ``needs_autoescape`` attributes are +``False``. You do not need to specify them if ``False`` is an acceptable +value. + Writing custom template tags ---------------------------- @@ -840,6 +929,43 @@ Ultimately, this decoupling of compilation and rendering results in an efficient template system, because a template can render multiple context without having to be parsed multiple times. +Auto-escaping considerations +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The output from template tags is not automatically run through the +auto-escaping filters. However, there are still a couple of things you should +keep in mind when writing a template tag: + +If the ``render()`` function of your template stores the result in a context +variable (rather than returning the result in a string), it should take care +to call ``mark_safe()`` if appropriate. When the variable is ultimately +rendered, it will be affected by the auto-escape setting in effect at the +time, so content that should be safe from further escaping needs to be marked +as such. + +Also, if your template tag creates a new context for performing some +sub-rendering, you should be careful to set the auto-escape attribute to the +current context's value. The ``__init__`` method for the ``Context`` class +takes a parameter called ``autoescape`` that you can use for this purpose. For +example:: + + def render(self, context): + # ... + new_context = Context({'var': obj}, autoescape=context.autoescape) + # ... Do something with new_context ... + +This is not a very common situation, but it is sometimes useful, particularly +if you are rendering a template yourself. For example:: + + def render(self, context): + t = template.load_template('small_fragment.html') + return t.render(Context({'var': obj}, autoescape=context.autoescape)) + +If we had neglected to pass in the current ``context.autoescape`` value to our +new ``Context`` in this example, the results would have *always* been +automatically escaped, which may not be the desired behaviour if the template +tag is used inside a ``{% autoescape off %}`` block. + Registering the tag ~~~~~~~~~~~~~~~~~~~ @@ -917,7 +1043,7 @@ current context, available in the ``render`` method:: def __init__(self, date_to_be_formatted, format_string): self.date_to_be_formatted = date_to_be_formatted self.format_string = format_string - + def render(self, context): try: actual_date = resolve_variable(self.date_to_be_formatted, context) @@ -934,26 +1060,26 @@ format it accordingly. ``template.resolve_variable()`` is still available, but has been deprecated in favor of a new ``template.Variable`` class. Using this class will usually be more efficient than calling ``template.resolve_variable`` - + To use the ``Variable`` class, simply instantiate it with the name of the variable to be resolved, and then call ``variable.resolve(context)``. So, in the development version, the above example would be more correctly written as: - + .. parsed-literal:: - + class FormatTimeNode(template.Node): def __init__(self, date_to_be_formatted, format_string): self.date_to_be_formatted = **Variable(date_to_be_formatted)** self.format_string = format_string - + def render(self, context): try: actual_date = **self.date_to_be_formatted.resolve(context)** return actual_date.strftime(self.format_string) except template.VariableDoesNotExist: return '' - + Changes are highlighted in bold. Variable resolution will throw a ``VariableDoesNotExist`` exception if it cannot |