diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/internals/deprecation.txt | 19 | ||||
| -rw-r--r-- | docs/ref/databases.txt | 59 | ||||
| -rw-r--r-- | docs/ref/middleware.txt | 4 | ||||
| -rw-r--r-- | docs/ref/request-response.txt | 4 | ||||
| -rw-r--r-- | docs/ref/settings.txt | 30 | ||||
| -rw-r--r-- | docs/releases/1.3-alpha-1.txt | 6 | ||||
| -rw-r--r-- | docs/releases/1.3.txt | 6 | ||||
| -rw-r--r-- | docs/releases/1.6.txt | 37 | ||||
| -rw-r--r-- | docs/topics/db/sql.txt | 61 | ||||
| -rw-r--r-- | docs/topics/db/transactions.txt | 734 |
10 files changed, 653 insertions, 307 deletions
diff --git a/docs/internals/deprecation.txt b/docs/internals/deprecation.txt index b5173af298..19675801e4 100644 --- a/docs/internals/deprecation.txt +++ b/docs/internals/deprecation.txt @@ -329,6 +329,15 @@ these changes. 1.8 --- +* The following transaction management APIs will be removed: + + - ``TransactionMiddleware``, + - the decorators and context managers ``autocommit``, ``commit_on_success``, + and ``commit_manually``, + - the ``TRANSACTIONS_MANAGED`` setting. + + Upgrade paths are described in :ref:`transactions-upgrading-from-1.5`. + * The :ttag:`cycle` and :ttag:`firstof` template tags will auto-escape their arguments. In 1.6 and 1.7, this behavior is provided by the version of these tags in the ``future`` template tag library. @@ -339,8 +348,6 @@ these changes. * ``Model._meta.module_name`` was renamed to ``model_name``. -* The private API ``django.db.close_connection`` will be removed. - * Remove the backward compatible shims introduced to rename ``get_query_set`` and similar queryset methods. This affects the following classes: ``BaseModelAdmin``, ``ChangeList``, ``BaseCommentNode``, @@ -350,6 +357,14 @@ these changes. * Remove the backward compatible shims introduced to rename the attributes ``ChangeList.root_query_set`` and ``ChangeList.query_set``. +* The following private APIs will be removed: + - ``django.db.close_connection()`` + - ``django.db.backends.creation.BaseDatabaseCreation.set_autocommit()`` + - ``django.db.transaction.is_managed()`` + - ``django.db.transaction.managed()`` + - ``django.db.transaction.commit_unless_managed()`` + - ``django.db.transaction.rollback_unless_managed()`` + 2.0 --- diff --git a/docs/ref/databases.txt b/docs/ref/databases.txt index 4e435949a2..78c1bb3dda 100644 --- a/docs/ref/databases.txt +++ b/docs/ref/databases.txt @@ -69,7 +69,6 @@ even ``0``, because it doesn't make sense to maintain a connection that's unlikely to be reused. This will help keep the number of simultaneous connections to this database small. - The development server creates a new thread for each request it handles, negating the effect of persistent connections. @@ -104,7 +103,8 @@ Optimizing PostgreSQL's configuration Django needs the following parameters for its database connections: - ``client_encoding``: ``'UTF8'``, -- ``default_transaction_isolation``: ``'read committed'``, +- ``default_transaction_isolation``: ``'read committed'`` by default, + or the value set in the connection options (see below), - ``timezone``: ``'UTC'`` when :setting:`USE_TZ` is ``True``, value of :setting:`TIME_ZONE` otherwise. @@ -118,30 +118,16 @@ will do some additional queries to set these parameters. .. _ALTER ROLE: http://www.postgresql.org/docs/current/interactive/sql-alterrole.html -Transaction handling ---------------------- - -:doc:`By default </topics/db/transactions>`, Django runs with an open -transaction which it commits automatically when any built-in, data-altering -model function is called. The PostgreSQL backends normally operate the same as -any other Django backend in this respect. - .. _postgresql-autocommit-mode: Autocommit mode -~~~~~~~~~~~~~~~ +--------------- -If your application is particularly read-heavy and doesn't make many -database writes, the overhead of a constantly open transaction can -sometimes be noticeable. For those situations, you can configure Django -to use *"autocommit"* behavior for the connection, meaning that each database -operation will normally be in its own transaction, rather than having -the transaction extend over multiple operations. In this case, you can -still manually start a transaction if you're doing something that -requires consistency across multiple database operations. The -autocommit behavior is enabled by setting the ``autocommit`` key in -the :setting:`OPTIONS` part of your database configuration in -:setting:`DATABASES`:: +.. versionchanged:: 1.6 + +In previous versions of Django, database-level autocommit could be enabled by +setting the ``autocommit`` key in the :setting:`OPTIONS` part of your database +configuration in :setting:`DATABASES`:: DATABASES = { # ... @@ -150,29 +136,11 @@ the :setting:`OPTIONS` part of your database configuration in }, } -In this configuration, Django still ensures that :ref:`delete() -<topics-db-queries-delete>` and :ref:`update() <topics-db-queries-update>` -queries run inside a single transaction, so that either all the affected -objects are changed or none of them are. - -.. admonition:: This is database-level autocommit - - This functionality is not the same as the :ref:`autocommit - <topics-db-transactions-autocommit>` decorator. That decorator is - a Django-level implementation that commits automatically after - data changing operations. The feature enabled using the - :setting:`OPTIONS` option provides autocommit behavior at the - database adapter level. It commits after *every* operation. - -If you are using this feature and performing an operation akin to delete or -updating that requires multiple operations, you are strongly recommended to -wrap you operations in manual transaction handling to ensure data consistency. -You should also audit your existing code for any instances of this behavior -before enabling this feature. It's faster, but it provides less automatic -protection for multi-call operations. +Since Django 1.6, autocommit is turned on by default. This configuration is +ignored and can be safely removed. Isolation level -~~~~~~~~~~~~~~~ +--------------- .. versionadded:: 1.6 @@ -200,7 +168,7 @@ such as ``REPEATABLE READ`` or ``SERIALIZABLE``, set it in the .. _postgresql-isolation-levels: http://www.postgresql.org/docs/devel/static/transaction-iso.html Indexes for ``varchar`` and ``text`` columns -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +-------------------------------------------- When specifying ``db_index=True`` on your model fields, Django typically outputs a single ``CREATE INDEX`` statement. However, if the database type @@ -456,8 +424,7 @@ Savepoints Both the Django ORM and MySQL (when using the InnoDB :ref:`storage engine <mysql-storage-engines>`) support database :ref:`savepoints -<topics-db-transactions-savepoints>`, but this feature wasn't available in -Django until version 1.4 when such supports was added. +<topics-db-transactions-savepoints>`. If you use the MyISAM storage engine please be aware of the fact that you will receive database-generated errors if you try to use the :ref:`savepoint-related diff --git a/docs/ref/middleware.txt b/docs/ref/middleware.txt index 1e6e57f720..20bb2fb751 100644 --- a/docs/ref/middleware.txt +++ b/docs/ref/middleware.txt @@ -205,6 +205,10 @@ Transaction middleware .. class:: TransactionMiddleware +.. versionchanged:: 1.6 + ``TransactionMiddleware`` is deprecated. The documentation of transactions + contains :ref:`upgrade instructions <transactions-upgrading-from-1.5>`. + Binds commit and rollback of the default database to the request/response phase. If a view function runs successfully, a commit is done. If it fails with an exception, a rollback is done. diff --git a/docs/ref/request-response.txt b/docs/ref/request-response.txt index 30f5e87100..6f620e17e2 100644 --- a/docs/ref/request-response.txt +++ b/docs/ref/request-response.txt @@ -814,8 +814,8 @@ generating large CSV files. .. admonition:: Performance considerations Django is designed for short-lived requests. Streaming responses will tie - a worker process and keep a database connection idle in transaction for - the entire duration of the response. This may result in poor performance. + a worker process for the entire duration of the response. This may result + in poor performance. Generally speaking, you should perform expensive tasks outside of the request-response cycle, rather than resorting to a streamed response. diff --git a/docs/ref/settings.txt b/docs/ref/settings.txt index 0cd141bcef..2b80527d8b 100644 --- a/docs/ref/settings.txt +++ b/docs/ref/settings.txt @@ -408,6 +408,30 @@ SQLite. This can be configured using the following:: For other database backends, or more complex SQLite configurations, other options will be required. The following inner options are available. +.. setting:: DATABASE-ATOMIC_REQUESTS + +ATOMIC_REQUESTS +~~~~~~~~~~~~~~~ + +.. versionadded:: 1.6 + +Default: ``False`` + +Set this to ``True`` to wrap each HTTP request in a transaction on this +database. See :ref:`tying-transactions-to-http-requests`. + +.. setting:: DATABASE-AUTOCOMMIT + +AUTOCOMMIT +~~~~~~~~~~ + +.. versionadded:: 1.6 + +Default: ``True`` + +Set this to ``False`` if you want to :ref:`disable Django's transaction +management <deactivate-transaction-management>` and implement your own. + .. setting:: DATABASE-ENGINE ENGINE @@ -1807,6 +1831,12 @@ to ensure your processes are running in the correct environment. TRANSACTIONS_MANAGED -------------------- +.. deprecated:: 1.6 + + This setting was deprecated because its name is very misleading. Use the + :setting:`AUTOCOMMIT <DATABASE-AUTOCOMMIT>` key in :setting:`DATABASES` + entries instead. + Default: ``False`` Set this to ``True`` if you want to :ref:`disable Django's transaction diff --git a/docs/releases/1.3-alpha-1.txt b/docs/releases/1.3-alpha-1.txt index ba8a4fc557..53d38a006b 100644 --- a/docs/releases/1.3-alpha-1.txt +++ b/docs/releases/1.3-alpha-1.txt @@ -105,16 +105,14 @@ you just won't get any of the nice new unittest2 features. Transaction context managers ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Users of Python 2.5 and above may now use :ref:`transaction management functions -<transaction-management-functions>` as `context managers`_. For example:: +Users of Python 2.5 and above may now use transaction management functions as +`context managers`_. For example:: with transaction.autocommit(): # ... .. _context managers: http://docs.python.org/glossary.html#term-context-manager -For more information, see :ref:`transaction-management-functions`. - Configurable delete-cascade ~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/docs/releases/1.3.txt b/docs/releases/1.3.txt index 4c8dd2f81f..582bceffca 100644 --- a/docs/releases/1.3.txt +++ b/docs/releases/1.3.txt @@ -148,16 +148,14 @@ you just won't get any of the nice new unittest2 features. Transaction context managers ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Users of Python 2.5 and above may now use :ref:`transaction management functions -<transaction-management-functions>` as `context managers`_. For example:: +Users of Python 2.5 and above may now use transaction management functions as +`context managers`_. For example:: with transaction.autocommit(): # ... .. _context managers: http://docs.python.org/glossary.html#term-context-manager -For more information, see :ref:`transaction-management-functions`. - Configurable delete-cascade ~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/docs/releases/1.6.txt b/docs/releases/1.6.txt index d78d594c90..a1fe69229c 100644 --- a/docs/releases/1.6.txt +++ b/docs/releases/1.6.txt @@ -30,6 +30,18 @@ prevention <clickjacking-prevention>` are turned on. If the default templates don't suit your tastes, you can use :ref:`custom project and app templates <custom-app-and-project-templates>`. +Improved transaction management +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Django's transaction management was overhauled. Database-level autocommit is +now turned on by default. This makes transaction handling more explicit and +should improve performance. The existing APIs were deprecated, and new APIs +were introduced, as described in :doc:`/topics/db/transactions`. + +Please review carefully the list of :ref:`known backwards-incompatibilities +<transactions-upgrading-from-1.5>` to determine if you need to make changes in +your code. + Persistent database connections ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -148,6 +160,16 @@ Backwards incompatible changes in 1.6 deprecation timeline for a given feature, its removal may appear as a backwards incompatible change. +* Database-level autocommit is enabled by default in Django 1.6. While this + doesn't change the general spirit of Django's transaction management, there + are a few known backwards-incompatibities, described in the :ref:`transaction + management docs <transactions-upgrading-from-1.5>`. You should review your code + to determine if you're affected. + +* In previous versions, database-level autocommit was only an option for + PostgreSQL, and it was disabled by default. This option is now + :ref:`ignored <postgresql-autocommit-mode>`. + * The ``django.db.models.query.EmptyQuerySet`` can't be instantiated any more - it is only usable as a marker class for checking if :meth:`~django.db.models.query.QuerySet.none` has been called: @@ -234,6 +256,21 @@ Backwards incompatible changes in 1.6 Features deprecated in 1.6 ========================== +Transaction management APIs +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Transaction management was completely overhauled in Django 1.6, and the +current APIs are deprecated: + +- ``django.middleware.transaction.TransactionMiddleware`` +- ``django.db.transaction.autocommit`` +- ``django.db.transaction.commit_on_success`` +- ``django.db.transaction.commit_manually`` +- the ``TRANSACTIONS_MANAGED`` setting + +The reasons for this change and the upgrade path are described in the +:ref:`transactions documentation <transactions-upgrading-from-1.5>`. + Changes to :ttag:`cycle` and :ttag:`firstof` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/docs/topics/db/sql.txt b/docs/topics/db/sql.txt index 6cc174a248..b2161fe65b 100644 --- a/docs/topics/db/sql.txt +++ b/docs/topics/db/sql.txt @@ -201,31 +201,32 @@ perform queries that don't map cleanly to models, or directly execute In these cases, you can always access the database directly, routing around the model layer entirely. -The object ``django.db.connection`` represents the -default database connection, and ``django.db.transaction`` represents the -default database transaction. To use the database connection, call -``connection.cursor()`` to get a cursor object. Then, call -``cursor.execute(sql, [params])`` to execute the SQL and ``cursor.fetchone()`` -or ``cursor.fetchall()`` to return the resulting rows. After performing a data -changing operation, you should then call -``transaction.commit_unless_managed()`` to ensure your changes are committed -to the database. If your query is purely a data retrieval operation, no commit -is required. For example:: +The object ``django.db.connection`` represents the default database +connection. To use the database connection, call ``connection.cursor()`` to +get a cursor object. Then, call ``cursor.execute(sql, [params])`` to execute +the SQL and ``cursor.fetchone()`` or ``cursor.fetchall()`` to return the +resulting rows. + +For example:: + + from django.db import connection def my_custom_sql(): - from django.db import connection, transaction cursor = connection.cursor() - # Data modifying operation - commit required cursor.execute("UPDATE bar SET foo = 1 WHERE baz = %s", [self.baz]) - transaction.commit_unless_managed() - # Data retrieval operation - no commit required cursor.execute("SELECT foo FROM bar WHERE baz = %s", [self.baz]) row = cursor.fetchone() return row +.. versionchanged:: 1.6 + In Django 1.5 and earlier, after performing a data changing operation, you + had to call ``transaction.commit_unless_managed()`` to ensure your changes + were committed to the database. Since Django now defaults to database-level + autocommit, this isn't necessary any longer. + If you are using :doc:`more than one database </topics/db/multi-db>`, you can use ``django.db.connections`` to obtain the connection (and cursor) for a specific database. ``django.db.connections`` is a dictionary-like @@ -235,7 +236,6 @@ alias:: from django.db import connections cursor = connections['my_db_alias'].cursor() # Your code here... - transaction.commit_unless_managed(using='my_db_alias') By default, the Python DB API will return results without their field names, which means you end up with a ``list`` of values, rather than a @@ -260,27 +260,18 @@ Here is an example of the difference between the two:: >>> dictfetchall(cursor) [{'parent_id': None, 'id': 54360982L}, {'parent_id': None, 'id': 54360880L}] - -.. _transactions-and-raw-sql: - -Transactions and raw SQL ------------------------- - -When you make a raw SQL call, Django will automatically mark the -current transaction as dirty. You must then ensure that the -transaction containing those calls is closed correctly. See :ref:`the -notes on the requirements of Django's transaction handling -<topics-db-transactions-requirements>` for more details. - Connections and cursors ----------------------- ``connection`` and ``cursor`` mostly implement the standard Python DB-API -described in :pep:`249` (except when it comes to :doc:`transaction handling -</topics/db/transactions>`). If you're not familiar with the Python DB-API, note -that the SQL statement in ``cursor.execute()`` uses placeholders, ``"%s"``, -rather than adding parameters directly within the SQL. If you use this -technique, the underlying database library will automatically add quotes and -escaping to your parameter(s) as necessary. (Also note that Django expects the -``"%s"`` placeholder, *not* the ``"?"`` placeholder, which is used by the SQLite -Python bindings. This is for the sake of consistency and sanity.) +described in :pep:`249` — except when it comes to :doc:`transaction handling +</topics/db/transactions>`. + +If you're not familiar with the Python DB-API, note that the SQL statement in +``cursor.execute()`` uses placeholders, ``"%s"``, rather than adding +parameters directly within the SQL. If you use this technique, the underlying +database library will automatically escape your parameters as necessary. + +Also note that Django expects the ``"%s"`` placeholder, *not* the ``"?"`` +placeholder, which is used by the SQLite Python bindings. This is for the sake +of consistency and sanity. diff --git a/docs/topics/db/transactions.txt b/docs/topics/db/transactions.txt index 11755ff5c5..b8017e8bfa 100644 --- a/docs/topics/db/transactions.txt +++ b/docs/topics/db/transactions.txt @@ -1,286 +1,375 @@ -============================== -Managing database transactions -============================== +===================== +Database transactions +===================== .. module:: django.db.transaction -Django gives you a few ways to control how database transactions are managed, -if you're using a database that supports transactions. +Django gives you a few ways to control how database transactions are managed. + +Managing database transactions +============================== Django's default transaction behavior -===================================== +------------------------------------- + +Django's default behavior is to run in autocommit mode. Each query is +immediately committed to the database. :ref:`See below for details +<autocommit-details>`. + +Django uses transactions or savepoints automatically to guarantee the +integrity of ORM operations that require multiple queries, especially +:ref:`delete() <topics-db-queries-delete>` and :ref:`update() +<topics-db-queries-update>` queries. -Django's default behavior is to run with an open transaction which it -commits automatically when any built-in, data-altering model function is -called. For example, if you call ``model.save()`` or ``model.delete()``, the -change will be committed immediately. +.. versionchanged:: 1.6 + Previous version of Django featured :ref:`a more complicated default + behavior <transactions-upgrading-from-1.5>`. -This is much like the auto-commit setting for most databases. As soon as you -perform an action that needs to write to the database, Django produces the -``INSERT``/``UPDATE``/``DELETE`` statements and then does the ``COMMIT``. -There's no implicit ``ROLLBACK``. +.. _tying-transactions-to-http-requests: Tying transactions to HTTP requests -=================================== +----------------------------------- -The recommended way to handle transactions in Web requests is to tie them to -the request and response phases via Django's ``TransactionMiddleware``. +A common way to handle transactions on the web is to wrap each request in a +transaction. Set :setting:`ATOMIC_REQUESTS <DATABASE-ATOMIC_REQUESTS>` to +``True`` in the configuration of each database for which you want to enable +this behavior. -It works like this: When a request starts, Django starts a transaction. If the -response is produced without problems, Django commits any pending transactions. -If the view function produces an exception, Django rolls back any pending -transactions. +It works like this. When a request starts, Django starts a transaction. If the +response is produced without problems, Django commits the transaction. If the +view function produces an exception, Django rolls back the transaction. +Middleware always runs outside of this transaction. -To activate this feature, just add the ``TransactionMiddleware`` middleware to -your :setting:`MIDDLEWARE_CLASSES` setting:: +You may perfom partial commits and rollbacks in your view code, typically with +the :func:`atomic` context manager. However, at the end of the view, either +all the changes will be committed, or none of them. - MIDDLEWARE_CLASSES = ( - 'django.middleware.cache.UpdateCacheMiddleware', - 'django.contrib.sessions.middleware.SessionMiddleware', - 'django.middleware.common.CommonMiddleware', - 'django.middleware.transaction.TransactionMiddleware', - 'django.middleware.cache.FetchFromCacheMiddleware', - ) +To disable this behavior for a specific view, you must set the +``transactions_per_request`` attribute of the view function itself to +``False``, like this:: -The order is quite important. The transaction middleware applies not only to -view functions, but also for all middleware modules that come after it. So if -you use the session middleware after the transaction middleware, session -creation will be part of the transaction. + def my_view(request): + do_stuff() + my_view.transactions_per_request = False -The various cache middlewares are an exception: -``CacheMiddleware``, :class:`~django.middleware.cache.UpdateCacheMiddleware`, -and :class:`~django.middleware.cache.FetchFromCacheMiddleware` are never -affected. Even when using database caching, Django's cache backend uses its own -database cursor (which is mapped to its own database connection internally). +.. warning:: -.. note:: + While the simplicity of this transaction model is appealing, it also makes it + inefficient when traffic increases. Opening a transaction for every view has + some overhead. The impact on performance depends on the query patterns of your + application and on how well your database handles locking. - The ``TransactionMiddleware`` only affects the database aliased - as "default" within your :setting:`DATABASES` setting. If you are using - multiple databases and want transaction control over databases other than - "default", you will need to write your own transaction middleware. +.. admonition:: Per-request transactions and streaming responses -.. _transaction-management-functions: + When a view returns a :class:`~django.http.StreamingHttpResponse`, reading + the contents of the response will often execute code to generate the + content. Since the view has already returned, such code runs outside of + the transaction. -Controlling transaction management in views -=========================================== + Generally speaking, it isn't advisable to write to the database while + generating a streaming response, since there's no sensible way to handle + errors after starting to send the response. -For most people, implicit request-based transactions work wonderfully. However, -if you need more fine-grained control over how transactions are managed, you can -use a set of functions in ``django.db.transaction`` to control transactions on a -per-function or per-code-block basis. +In practice, this feature simply wraps every view function in the :func:`atomic` +decorator described below. -These functions, described in detail below, can be used in two different ways: +Note that only the execution of your view in enclosed in the transactions. +Middleware run outside of the transaction, and so does the rendering of +template responses. -* As a decorator_ on a particular function. For example:: +.. versionchanged:: 1.6 + Django used to provide this feature via ``TransactionMiddleware``, which is + now deprecated. - from django.db import transaction +Controlling transactions explicitly +----------------------------------- - @transaction.commit_on_success - def viewfunc(request): - # ... - # this code executes inside a transaction - # ... +.. versionadded:: 1.6 -* As a `context manager`_ around a particular block of code:: +Django provides a single API to control database transactions. - from django.db import transaction +.. function:: atomic(using=None, savepoint=True) - def viewfunc(request): - # ... - # this code executes using default transaction management - # ... + This function creates an atomic block for writes to the database. + (Atomicity is the defining property of database transactions.) - with transaction.commit_on_success(): - # ... - # this code executes inside a transaction - # ... + When the block completes successfully, the changes are committed to the + database. When it raises an exception, the changes are rolled back. -Both techniques work with all supported version of Python. + ``atomic`` can be nested. In this case, when an inner block completes + successfully, its effects can still be rolled back if an exception is + raised in the outer block at a later point. -.. _decorator: http://docs.python.org/glossary.html#term-decorator -.. _context manager: http://docs.python.org/glossary.html#term-context-manager + ``atomic`` takes a ``using`` argument which should be the name of a + database. If this argument isn't provided, Django uses the ``"default"`` + database. -For maximum compatibility, all of the examples below show transactions using the -decorator syntax, but all of the follow functions may be used as context -managers, too. + ``atomic`` is usable both as a `decorator`_:: -.. note:: + from django.db import transaction - Although the examples below use view functions as examples, these - decorators and context managers can be used anywhere in your code - that you need to deal with transactions. + @transaction.atomic + def viewfunc(request): + # This code executes inside a transaction. + do_stuff() -.. _topics-db-transactions-autocommit: + and as a `context manager`_:: -.. function:: autocommit + from django.db import transaction + + def viewfunc(request): + # This code executes in autocommit mode (Django's default). + do_stuff() - Use the ``autocommit`` decorator to switch a view function to Django's - default commit behavior, regardless of the global transaction setting. + with transaction.atomic(): + # This code executes inside a transaction. + do_more_stuff() - Example:: + .. _decorator: http://docs.python.org/glossary.html#term-decorator + .. _context manager: http://docs.python.org/glossary.html#term-context-manager - from django.db import transaction + Wrapping ``atomic`` in a try/except block allows for natural handling of + integrity errors:: + + from django.db import IntegrityError, transaction - @transaction.autocommit + @transaction.atomic def viewfunc(request): - .... + do_stuff() - @transaction.autocommit(using="my_other_database") - def viewfunc2(request): - .... + try: + with transaction.atomic(): + do_stuff_that_could_fail() + except IntegrityError: + handle_exception() - Within ``viewfunc()``, transactions will be committed as soon as you call - ``model.save()``, ``model.delete()``, or any other function that writes to - the database. ``viewfunc2()`` will have this same behavior, but for the - ``"my_other_database"`` connection. + do_more_stuff() -.. function:: commit_on_success + In this example, even if ``do_stuff_that_could_fail()`` causes a database + error by breaking an integrity constraint, you can execute queries in + ``do_more_stuff()``, and the changes from ``do_stuff()`` are still there. - Use the ``commit_on_success`` decorator to use a single transaction for all - the work done in a function:: + In order to guarantee atomicity, ``atomic`` disables some APIs. Attempting + to commit, roll back, or change the autocommit state of the database + connection within an ``atomic`` block will raise an exception. - from django.db import transaction + ``atomic`` can only be used in autocommit mode. It will raise an exception + if autocommit is turned off. - @transaction.commit_on_success - def viewfunc(request): - .... + Under the hood, Django's transaction management code: - @transaction.commit_on_success(using="my_other_database") - def viewfunc2(request): - .... + - opens a transaction when entering the outermost ``atomic`` block; + - creates a savepoint when entering an inner ``atomic`` block; + - releases or rolls back to the savepoint when exiting an inner block; + - commits or rolls back the transaction when exiting the outermost block. - If the function returns successfully, then Django will commit all work done - within the function at that point. If the function raises an exception, - though, Django will roll back the transaction. + You can disable the creation of savepoints for inner blocks by setting the + ``savepoint`` argument to ``False``. If an exception occurs, Django will + perform the rollback when exiting the first parent block with a savepoint + if there is one, and the outermost block otherwise. Atomicity is still + guaranteed by the outer transaction. This option should only be used if + the overhead of savepoints is noticeable. It has the drawback of breaking + the error handling described above. -.. function:: commit_manually +.. admonition:: Performance considerations - Use the ``commit_manually`` decorator if you need full control over - transactions. It tells Django you'll be managing the transaction on your - own. + Open transactions have a performance cost for your database server. To + minimize this overhead, keep your transactions as short as possible. This + is especially important of you're using :func:`atomic` in long-running + processes, outside of Django's request / response cycle. - Whether you are writing or simply reading from the database, you must - ``commit()`` or ``rollback()`` explicitly or Django will raise a - :exc:`TransactionManagementError` exception. This is required when reading - from the database because ``SELECT`` statements may call functions which - modify tables, and thus it is impossible to know if any data has been - modified. +Autocommit +========== - Manual transaction management looks like this:: +.. _autocommit-details: - from django.db import transaction +Why Django uses autocommit +-------------------------- - @transaction.commit_manually - def viewfunc(request): - ... - # You can commit/rollback however and whenever you want - transaction.commit() - ... +In the SQL standards, each SQL query starts a transaction, unless one is +already in progress. Such transactions must then be committed or rolled back. - # But you've got to remember to do it yourself! - try: - ... - except: - transaction.rollback() - else: - transaction.commit() +This isn't always convenient for application developers. To alleviate this +problem, most databases provide an autocommit mode. When autocommit is turned +on, each SQL query is wrapped in its own transaction. In other words, the +transaction is not only automatically started, but also automatically +committed. - @transaction.commit_manually(using="my_other_database") - def viewfunc2(request): - .... +:pep:`249`, the Python Database API Specification v2.0, requires autocommit to +be initially turned off. Django overrides this default and turns autocommit +on. -.. _topics-db-transactions-requirements: +To avoid this, you can :ref:`deactivate the transaction management +<deactivate-transaction-management>`, but it isn't recommended. -Requirements for transaction handling -===================================== +.. versionchanged:: 1.6 + Before Django 1.6, autocommit was turned off, and it was emulated by + forcing a commit after write operations in the ORM. -Django requires that every transaction that is opened is closed before -the completion of a request. If you are using :func:`autocommit` (the -default commit mode) or :func:`commit_on_success`, this will be done -for you automatically (with the exception of :ref:`executing custom SQL -<executing-custom-sql>`). However, if you are manually managing -transactions (using the :func:`commit_manually` decorator), you must -ensure that the transaction is either committed or rolled back before -a request is completed. +.. warning:: -This applies to all database operations, not just write operations. Even -if your transaction only reads from the database, the transaction must -be committed or rolled back before you complete a request. + If you're using the database API directly — for instance, you're running + SQL queries with ``cursor.execute()`` — be aware that autocommit is on, + and consider wrapping your operations in a transaction, with + :func:`atomic`, to ensure consistency. .. _deactivate-transaction-management: -How to globally deactivate transaction management -================================================= +Deactivating transaction management +----------------------------------- + +You can totally disable Django's transaction management for a given database +by setting :setting:`AUTOCOMMIT <DATABASE-AUTOCOMMIT>` to ``False`` in its +configuration. If you do this, Django won't enable autocommit, and won't +perform any commits. You'll get the regular behavior of the underlying +database library. + +This requires you to commit explicitly every transaction, even those started +by Django or by third-party libraries. Thus, this is best used in situations +where you want to run your own transaction-controlling middleware or do +something really strange. + +.. versionchanged:: 1.6 + This used to be controlled by the ``TRANSACTIONS_MANAGED`` setting. + +Low-level APIs +============== + +.. warning:: + + Always prefer :func:`atomic` if possible at all. It accounts for the + idiosyncrasies of each database and prevents invalid operations. + + The low level APIs are only useful if you're implementing your own + transaction management. + +.. _managing-autocommit: + +Autocommit +---------- + +.. versionadded:: 1.6 + +Django provides a straightforward API to manage the autocommit state of each +database connection, if you need to. + +.. function:: get_autocommit(using=None) + +.. function:: set_autocommit(autocommit, using=None) + +These functions take a ``using`` argument which should be the name of a +database. If it isn't provided, Django uses the ``"default"`` database. + +Autocommit is initially turned on. If you turn it off, it's your +responsibility to restore it. + +Once you turn autocommit off, you get the default behavior of your database +adapter, and Django won't help you. Although that behavior is specified in +:pep:`249`, implementations of adapters aren't always consistent with one +another. Review the documentation of the adapter you're using carefully. + +You must ensure that no transaction is active, usually by issuing a +:func:`commit` or a :func:`rollback`, before turning autocommit back on. + +:func:`atomic` requires autocommit to be turned on; it will raise an exception +if autocommit is off. Django will also refuse to turn autocommit off when an +:func:`atomic` block is active, because that would break atomicity. + +Transactions +------------ -Control freaks can totally disable all transaction management by setting -:setting:`TRANSACTIONS_MANAGED` to ``True`` in the Django settings file. +A transaction is an atomic set of database queries. Even if your program +crashes, the database guarantees that either all the changes will be applied, +or none of them. -If you do this, Django won't provide any automatic transaction management -whatsoever. Middleware will no longer implicitly commit transactions, and -you'll need to roll management yourself. This even requires you to commit -changes done by middleware somewhere else. +Django doesn't provide an API to start a transaction. The expected way to +start a transaction is to disable autocommit with :func:`set_autocommit`. -Thus, this is best used in situations where you want to run your own -transaction-controlling middleware or do something really strange. In almost -all situations, you'll be better off using the default behavior, or the -transaction middleware, and only modify selected functions as needed. +Once you're in a transaction, you can choose either to apply the changes +you've performed until this point with :func:`commit`, or to cancel them with +:func:`rollback`. + +.. function:: commit(using=None) + +.. function:: rollback(using=None) + +These functions take a ``using`` argument which should be the name of a +database. If it isn't provided, Django uses the ``"default"`` database. + +Django will refuse to commit or to rollback when an :func:`atomic` block is +active, because that would break atomicity. .. _topics-db-transactions-savepoints: Savepoints -========== +---------- -A savepoint is a marker within a transaction that enables you to roll back part -of a transaction, rather than the full transaction. Savepoints are available -with the PostgreSQL 8, Oracle and MySQL (when using the InnoDB storage engine) -backends. Other backends provide the savepoint functions, but they're empty -operations -- they don't actually do anything. +A savepoint is a marker within a transaction that enables you to roll back +part of a transaction, rather than the full transaction. Savepoints are +available with the SQLite (≥ 3.6.8), PostgreSQL, Oracle and MySQL (when using +the InnoDB storage engine) backends. Other backends provide the savepoint +functions, but they're empty operations -- they don't actually do anything. -Savepoints aren't especially useful if you are using the default -``autocommit`` behavior of Django. However, if you are using -``commit_on_success`` or ``commit_manually``, each open transaction will build -up a series of database operations, awaiting a commit or rollback. If you -issue a rollback, the entire transaction is rolled back. Savepoints provide -the ability to perform a fine-grained rollback, rather than the full rollback -that would be performed by ``transaction.rollback()``. +Savepoints aren't especially useful if you are using autocommit, the default +behavior of Django. However, once you open a transaction with :func:`atomic`, +you build up a series of database operations awaiting a commit or rollback. If +you issue a rollback, the entire transaction is rolled back. Savepoints +provide the ability to perform a fine-grained rollback, rather than the full +rollback that would be performed by ``transaction.rollback()``. + +.. versionchanged:: 1.6 + +When the :func:`atomic` decorator is nested, it creates a savepoint to allow +partial commit or rollback. You're strongly encouraged to use :func:`atomic` +rather than the functions described below, but they're still part of the +public API, and there's no plan to deprecate them. Each of these functions takes a ``using`` argument which should be the name of a database for which the behavior applies. If no ``using`` argument is provided then the ``"default"`` database is used. -Savepoints are controlled by three methods on the transaction object: +Savepoints are controlled by three functions in :mod:`django.db.transaction`: -.. method:: transaction.savepoint(using=None) +.. function:: savepoint(using=None) Creates a new savepoint. This marks a point in the transaction that is known to be in a "good" state. - Returns the savepoint ID (sid). + Returns the savepoint ID (``sid``). + +.. function:: savepoint_commit(sid, using=None) + + Releases savepoint ``sid``. The changes performed since the savepoint was + created become part of the transaction. + +.. function:: savepoint_rollback(sid, using=None) -.. method:: transaction.savepoint_commit(sid, using=None) + Rolls back the transaction to savepoint ``sid``. - Updates the savepoint to include any operations that have been performed - since the savepoint was created, or since the last commit. +These functions do nothing if savepoints aren't supported or if the database +is in autocommit mode. -.. method:: transaction.savepoint_rollback(sid, using=None) +In addition, there's a utility function: - Rolls the transaction back to the last point at which the savepoint was - committed. +.. function:: clean_savepoints(using=None) + + Resets the counter used to generate unique savepoint IDs. The following example demonstrates the use of savepoints:: from django.db import transaction - @transaction.commit_manually + # open a transaction + @transaction.atomic def viewfunc(request): a.save() - # open transaction now contains a.save() + # transaction now contains a.save() + sid = transaction.savepoint() b.save() - # open transaction now contains a.save() and b.save() + # transaction now contains a.save() and b.save() if want_to_keep_b: transaction.savepoint_commit(sid) @@ -289,10 +378,25 @@ The following example demonstrates the use of savepoints:: transaction.savepoint_rollback(sid) # open transaction now contains only a.save() - transaction.commit() +Database-specific notes +======================= + +Savepoints in SQLite +-------------------- + +While SQLite ≥ 3.6.8 supports savepoints, a flaw in the design of the +:mod:`sqlite3` makes them hardly usable. + +When autocommit is enabled, savepoints don't make sense. When it's disabled, +:mod:`sqlite3` commits implicitly before savepoint-related statement. (It +commits before any statement other than ``SELECT``, ``INSERT``, ``UPDATE``, +``DELETE`` and ``REPLACE``.) + +As a consequence, savepoints are only usable inside a transaction ie. inside +an :func:`atomic` block. Transactions in MySQL -===================== +--------------------- If you're using MySQL, your tables may or may not support transactions; it depends on your MySQL version and the table types you're using. (By @@ -301,14 +405,14 @@ peculiarities are outside the scope of this article, but the MySQL site has `information on MySQL transactions`_. If your MySQL setup does *not* support transactions, then Django will function -in auto-commit mode: Statements will be executed and committed as soon as +in autocommit mode: Statements will be executed and committed as soon as they're called. If your MySQL setup *does* support transactions, Django will handle transactions as explained in this document. .. _information on MySQL transactions: http://dev.mysql.com/doc/refman/5.0/en/sql-syntax-transactions.html Handling exceptions within PostgreSQL transactions -================================================== +-------------------------------------------------- When a call to a PostgreSQL cursor raises an exception (typically ``IntegrityError``), all subsequent SQL in the same transaction will fail with @@ -321,7 +425,7 @@ force_insert/force_update flag, or invoking custom SQL. There are several ways to recover from this sort of error. Transaction rollback --------------------- +~~~~~~~~~~~~~~~~~~~~ The first option is to roll back the entire transaction. For example:: @@ -338,13 +442,13 @@ made by ``a.save()`` would be lost, even though that operation raised no error itself. Savepoint rollback ------------------- +~~~~~~~~~~~~~~~~~~ -If you are using PostgreSQL 8 or later, you can use :ref:`savepoints -<topics-db-transactions-savepoints>` to control the extent of a rollback. -Before performing a database operation that could fail, you can set or update -the savepoint; that way, if the operation fails, you can roll back the single -offending operation, rather than the entire transaction. For example:: +You can use :ref:`savepoints <topics-db-transactions-savepoints>` to control +the extent of a rollback. Before performing a database operation that could +fail, you can set or update the savepoint; that way, if the operation fails, +you can roll back the single offending operation, rather than the entire +transaction. For example:: a.save() # Succeeds, and never undone by savepoint rollback try: @@ -358,25 +462,227 @@ offending operation, rather than the entire transaction. For example:: In this example, ``a.save()`` will not be undone in the case where ``b.save()`` raises an exception. -Database-level autocommit -------------------------- +.. _transactions-upgrading-from-1.5: -With PostgreSQL 8.2 or later, there is an advanced option to run PostgreSQL -with :doc:`database-level autocommit </ref/databases>`. If you use this option, -there is no constantly open transaction, so it is always possible to continue -after catching an exception. For example:: +Changes from Django 1.5 and earlier +=================================== + +The features described below were deprecated in Django 1.6 and will be removed +in Django 1.8. They're documented in order to ease the migration to the new +transaction management APIs. + +Legacy APIs +----------- + +The following functions, defined in ``django.db.transaction``, provided a way +to control transactions on a per-function or per-code-block basis. They could +be used as decorators or as context managers, and they accepted a ``using`` +argument, exactly like :func:`atomic`. + +.. function:: autocommit + + Enable Django's default autocommit behavior. + + Transactions will be committed as soon as you call ``model.save()``, + ``model.delete()``, or any other function that writes to the database. + +.. function:: commit_on_success + + Use a single transaction for all the work done in a function. + + If the function returns successfully, then Django will commit all work done + within the function at that point. If the function raises an exception, + though, Django will roll back the transaction. + +.. function:: commit_manually + + Tells Django you'll be managing the transaction on your own. + + Whether you are writing or simply reading from the database, you must + ``commit()`` or ``rollback()`` explicitly or Django will raise a + :exc:`TransactionManagementError` exception. This is required when reading + from the database because ``SELECT`` statements may call functions which + modify tables, and thus it is impossible to know if any data has been + modified. + +.. _transaction-states: + +Transaction states +------------------ + +The three functions described above relied on a concept called "transaction +states". This mechanisme was deprecated in Django 1.6, but it's still +available until Django 1.8.. + +At any time, each database connection is in one of these two states: + +- **auto mode**: autocommit is enabled; +- **managed mode**: autocommit is disabled. + +Django starts in auto mode. ``TransactionMiddleware``, +:func:`commit_on_success` and :func:`commit_manually` activate managed mode; +:func:`autocommit` activates auto mode. + +Internally, Django keeps a stack of states. Activations and deactivations must +be balanced. + +For example, ``commit_on_success`` switches to managed mode when entering the +block of code it controls; when exiting the block, it commits or rollbacks, +and switches back to auto mode. + +So :func:`commit_on_success` really has two effects: it changes the +transaction state and it defines an transaction block. Nesting will give the +expected results in terms of transaction state, but not in terms of +transaction semantics. Most often, the inner block will commit, breaking the +atomicity of the outer block. + +:func:`autocommit` and :func:`commit_manually` have similar limitations. + +API changes +----------- + +Transaction middleware +~~~~~~~~~~~~~~~~~~~~~~ + +In Django 1.6, ``TransactionMiddleware`` is deprecated and replaced +:setting:`ATOMIC_REQUESTS <DATABASE-ATOMIC_REQUESTS>`. While the general +behavior is the same, there are a few differences. + +With the transaction middleware, it was still possible to switch to autocommit +or to commit explicitly in a view. Since :func:`atomic` guarantees atomicity, +this isn't allowed any longer. - a.save() # succeeds +To avoid wrapping a particular view in a transaction, instead of:: + + @transaction.autocommit + def my_view(request): + do_stuff() + +you must now use this pattern:: + + def my_view(request): + do_stuff() + my_view.transactions_per_request = False + +The transaction middleware applied not only to view functions, but also to +middleware modules that come after it. For instance, if you used the session +middleware after the transaction middleware, session creation was part of the +transaction. :setting:`ATOMIC_REQUESTS <DATABASE-ATOMIC_REQUESTS>` only +applies to the view itself. + +Managing transactions +~~~~~~~~~~~~~~~~~~~~~ + +Starting with Django 1.6, :func:`atomic` is the only supported API for +defining a transaction. Unlike the deprecated APIs, it's nestable and always +guarantees atomicity. + +In most cases, it will be a drop-in replacement for :func:`commit_on_success`. + +During the deprecation period, it's possible to use :func:`atomic` within +:func:`autocommit`, :func:`commit_on_success` or :func:`commit_manually`. +However, the reverse is forbidden, because nesting the old decorators / +context managers breaks atomicity. + +If you enter :func:`atomic` while you're in managed mode, it will trigger a +commit to start from a clean slate. + +Managing autocommit +~~~~~~~~~~~~~~~~~~~ + +Django 1.6 introduces an explicit :ref:`API for mananging autocommit +<managing-autocommit>`. + +To disable autocommit temporarily, instead of:: + + with transaction.commit_manually(): + # do stuff + +you should now use:: + + transaction.set_autocommit(False) try: - b.save() # Could throw exception - except IntegrityError: - pass - c.save() # succeeds + # do stuff + finally: + transaction.set_autocommit(True) + +To enable autocommit temporarily, instead of:: + + with transaction.autocommit(): + # do stuff + +you should now use:: + + transaction.set_autocommit(True) + try: + # do stuff + finally: + transaction.set_autocommit(False) + +Disabling transaction management +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Instead of setting ``TRANSACTIONS_MANAGED = True``, set the ``AUTOCOMMIT`` key +to ``False`` in the configuration of each database, as explained in :ref +:`deactivate-transaction-management`. + +Backwards incompatibilities +--------------------------- + +Since version 1.6, Django uses database-level autocommit in auto mode. +Previously, it implemented application-level autocommit by triggering a commit +after each ORM write. + +As a consequence, each database query (for instance, an ORM read) started a +transaction that lasted until the next ORM write. Such "automatic +transactions" no longer exist in Django 1.6. + +There are four known scenarios where this is backwards-incompatible. + +Note that managed mode isn't affected at all. This section assumes auto mode. +See the :ref:`description of modes <transaction-states>` above. + +Sequences of custom SQL queries +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If you're executing several :ref:`custom SQL queries <executing-custom-sql>` +in a row, each one now runs in its own transaction, instead of sharing the +same "automatic transaction". If you need to enforce atomicity, you must wrap +the sequence of queries in :func:`commit_on_success`. + +To check for this problem, look for calls to ``cursor.execute()``. They're +usually followed by a call to ``transaction.commit_unless_managed``, which +isn't necessary any more and should be removed. + +Select for update +~~~~~~~~~~~~~~~~~ + +If you were relying on "automatic transactions" to provide locking between +:meth:`~django.db.models.query.QuerySet.select_for_update` and a subsequent +write operation — an extremely fragile design, but nonetheless possible — you +must wrap the relevant code in :func:`atomic`. + +Using a high isolation level +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If you were using the "repeatable read" isolation level or higher, and if you +relied on "automatic transactions" to guarantee consistency between successive +reads, the new behavior might be backwards-incompatible. To enforce +consistency, you must wrap such sequences in :func:`atomic`. + +MySQL defaults to "repeatable read" and SQLite to "serializable"; they may be +affected by this problem. + +At the "read committed" isolation level or lower, "automatic transactions" +have no effect on the semantics of any sequence of ORM operations. + +PostgreSQL and Oracle default to "read committed" and aren't affected, unless +you changed the isolation level. -.. note:: +Using unsupported database features +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - This is not the same as the :ref:`autocommit decorator - <topics-db-transactions-autocommit>`. When using database level autocommit - there is no database transaction at all. The ``autocommit`` decorator - still uses transactions, automatically committing each transaction when - a database modifying operation occurs. +With triggers, views, or functions, it's possible to make ORM reads result in +database modifications. Django 1.5 and earlier doesn't deal with this case and +it's theoretically possible to observe a different behavior after upgrading to +Django 1.6 or later. In doubt, use :func:`atomic` to enforce integrity. |