diff options
Diffstat (limited to 'docs/topics/testing')
| -rw-r--r-- | docs/topics/testing/advanced.txt | 77 | ||||
| -rw-r--r-- | docs/topics/testing/doctests.txt | 81 | ||||
| -rw-r--r-- | docs/topics/testing/index.txt | 77 | ||||
| -rw-r--r-- | docs/topics/testing/overview.txt | 94 |
4 files changed, 89 insertions, 240 deletions
diff --git a/docs/topics/testing/advanced.txt b/docs/topics/testing/advanced.txt index 5f2fa65bed..cefb770469 100644 --- a/docs/topics/testing/advanced.txt +++ b/docs/topics/testing/advanced.txt @@ -165,7 +165,7 @@ environment first. Django provides a convenience method to do this:: :func:`~django.test.utils.setup_test_environment` puts several Django features into modes that allow for repeatable testing, but does not create the test -databases; :func:`django.test.simple.DjangoTestSuiteRunner.setup_databases` +databases; :func:`django.test.runner.DiscoverRunner.setup_databases` takes care of that. The call to :func:`~django.test.utils.setup_test_environment` is made @@ -178,27 +178,27 @@ tests via Django's test runner. Using different testing frameworks ================================== -Clearly, :mod:`doctest` and :mod:`unittest` are not the only Python testing -frameworks. While Django doesn't provide explicit support for alternative -frameworks, it does provide a way to invoke tests constructed for an -alternative framework as if they were normal Django tests. +Clearly, :mod:`unittest` is not the only Python testing framework. While Django +doesn't provide explicit support for alternative frameworks, it does provide a +way to invoke tests constructed for an alternative framework as if they were +normal Django tests. When you run ``./manage.py test``, Django looks at the :setting:`TEST_RUNNER` setting to determine what to do. By default, :setting:`TEST_RUNNER` points to -``'django.test.simple.DjangoTestSuiteRunner'``. This class defines the default Django +``'django.test.runner.DiscoverRunner'``. This class defines the default Django testing behavior. This behavior involves: #. Performing global pre-test setup. -#. Looking for unit tests and doctests in the ``models.py`` and - ``tests.py`` files in each installed application. +#. Looking for tests in any file below the current directory whose name matches + the pattern ``test*.py``. #. Creating the test databases. #. Running ``syncdb`` to install models and initial data into the test databases. -#. Running the unit tests and doctests that are found. +#. Running the tests that were found. #. Destroying the test databases. @@ -215,15 +215,22 @@ process to satisfy whatever testing requirements you may have. Defining a test runner ---------------------- -.. currentmodule:: django.test.simple +.. currentmodule:: django.test.runner A test runner is a class defining a ``run_tests()`` method. Django ships -with a ``DjangoTestSuiteRunner`` class that defines the default Django +with a ``DiscoverRunner`` class that defines the default Django testing behavior. This class defines the ``run_tests()`` entry point, plus a selection of other methods that are used to by ``run_tests()`` to set up, execute and tear down the test suite. -.. class:: DjangoTestSuiteRunner(verbosity=1, interactive=True, failfast=True, **kwargs) +.. class:: DiscoverRunner(pattern='test*.py', top_level=None, verbosity=1, interactive=True, failfast=True, **kwargs) + + ``DiscoverRunner`` will search for tests in any file matching ``pattern``. + + ``top_level`` can be used to specify the directory containing your + top-level Python modules. Usually Django can figure this out automatically, + so it's not necessary to specify this option. If specified, it should + generally be the directory containing your ``manage.py`` file. ``verbosity`` determines the amount of notification and debug information that will be printed to the console; ``0`` is no output, ``1`` is normal @@ -238,11 +245,10 @@ set up, execute and tear down the test suite. If ``failfast`` is ``True``, the test suite will stop running after the first test failure is detected. - Django will, from time to time, extend the capabilities of - the test runner by adding new arguments. The ``**kwargs`` declaration - allows for this expansion. If you subclass ``DjangoTestSuiteRunner`` or - write your own test runner, ensure accept and handle the ``**kwargs`` - parameter. + Django may, from time to time, extend the capabilities of the test runner + by adding new arguments. The ``**kwargs`` declaration allows for this + expansion. If you subclass ``DiscoverRunner`` or write your own test + runner, ensure it accepts ``**kwargs``. Your test runner may also define additional command-line options. If you add an ``option_list`` attribute to a subclassed test runner, @@ -252,7 +258,7 @@ set up, execute and tear down the test suite. Attributes ~~~~~~~~~~ -.. attribute:: DjangoTestSuiteRunner.option_list +.. attribute:: DiscoverRunner.option_list This is the tuple of ``optparse`` options which will be fed into the management command's ``OptionParser`` for parsing arguments. See the @@ -261,20 +267,25 @@ Attributes Methods ~~~~~~~ -.. method:: DjangoTestSuiteRunner.run_tests(test_labels, extra_tests=None, **kwargs) +.. method:: DiscoverRunner.run_tests(test_labels, extra_tests=None, **kwargs) Run the test suite. ``test_labels`` is a list of strings describing the tests to be run. A test - label can take one of three forms: + label can take one of four forms: - * ``app.TestCase.test_method`` -- Run a single test method in a test + * ``path.to.test_module.TestCase.test_method`` -- Run a single test method + in a test case. + * ``path.to.test_module.TestCase`` -- Run all the test methods in a test case. - * ``app.TestCase`` -- Run all the test methods in a test case. - * ``app`` -- Search for and run all tests in the named application. + * ``path.to.module`` -- Search for and run all tests in the named Python + package or module. + * ``path/to/directory`` -- Search for and run all tests below the named + directory. - If ``test_labels`` has a value of ``None``, the test runner should run - search for tests in all the applications in :setting:`INSTALLED_APPS`. + If ``test_labels`` has a value of ``None``, the test runner will search for + tests in all files below the current directory whose names match its + ``pattern`` (see above). ``extra_tests`` is a list of extra ``TestCase`` instances to add to the suite that is executed by the test runner. These extra tests are run @@ -282,13 +293,13 @@ Methods This method should return the number of tests that failed. -.. method:: DjangoTestSuiteRunner.setup_test_environment(**kwargs) +.. method:: DiscoverRunner.setup_test_environment(**kwargs) Sets up the test environment by calling :func:`~django.test.utils.setup_test_environment` and setting :setting:`DEBUG` to ``False``. -.. method:: DjangoTestSuiteRunner.build_suite(test_labels, extra_tests=None, **kwargs) +.. method:: DiscoverRunner.build_suite(test_labels, extra_tests=None, **kwargs) Constructs a test suite that matches the test labels provided. @@ -309,7 +320,7 @@ Methods Returns a ``TestSuite`` instance ready to be run. -.. method:: DjangoTestSuiteRunner.setup_databases(**kwargs) +.. method:: DiscoverRunner.setup_databases(**kwargs) Creates the test databases. @@ -317,13 +328,13 @@ Methods that have been made. This data will be provided to the ``teardown_databases()`` function at the conclusion of testing. -.. method:: DjangoTestSuiteRunner.run_suite(suite, **kwargs) +.. method:: DiscoverRunner.run_suite(suite, **kwargs) Runs the test suite. Returns the result produced by the running the test suite. -.. method:: DjangoTestSuiteRunner.teardown_databases(old_config, **kwargs) +.. method:: DiscoverRunner.teardown_databases(old_config, **kwargs) Destroys the test databases, restoring pre-test conditions. @@ -331,11 +342,11 @@ Methods database configuration that need to be reversed. It is the return value of the ``setup_databases()`` method. -.. method:: DjangoTestSuiteRunner.teardown_test_environment(**kwargs) +.. method:: DiscoverRunner.teardown_test_environment(**kwargs) Restores the pre-test environment. -.. method:: DjangoTestSuiteRunner.suite_result(suite, result, **kwargs) +.. method:: DiscoverRunner.suite_result(suite, result, **kwargs) Computes and returns a return code based on a test suite, and the result from that test suite. @@ -402,7 +413,7 @@ can be useful during testing. ``old_database_name``. The ``verbosity`` argument has the same behavior as for - :class:`~django.test.simple.DjangoTestSuiteRunner`. + :class:`~django.test.runner.DiscoverRunner`. .. _topics-testing-code-coverage: diff --git a/docs/topics/testing/doctests.txt b/docs/topics/testing/doctests.txt deleted file mode 100644 index 5036e946a9..0000000000 --- a/docs/topics/testing/doctests.txt +++ /dev/null @@ -1,81 +0,0 @@ -=================== -Django and doctests -=================== - -Doctests use Python's standard :mod:`doctest` module, which searches your -docstrings for statements that resemble a session of the Python interactive -interpreter. A full explanation of how :mod:`doctest` works is out of the scope -of this document; read Python's official documentation for the details. - -.. admonition:: What's a **docstring**? - - A good explanation of docstrings (and some guidelines for using them - effectively) can be found in :pep:`257`: - - A docstring is a string literal that occurs as the first statement in - a module, function, class, or method definition. Such a docstring - becomes the ``__doc__`` special attribute of that object. - - For example, this function has a docstring that describes what it does:: - - def add_two(num): - "Return the result of adding two to the provided number." - return num + 2 - - Because tests often make great documentation, putting tests directly in - your docstrings is an effective way to document *and* test your code. - -As with unit tests, for a given Django application, the test runner looks for -doctests in two places: - -* The ``models.py`` file. You can define module-level doctests and/or a - doctest for individual models. It's common practice to put - application-level doctests in the module docstring and model-level - doctests in the model docstrings. - -* A file called ``tests.py`` in the application directory -- i.e., the - directory that holds ``models.py``. This file is a hook for any and all - doctests you want to write that aren't necessarily related to models. - -This example doctest is equivalent to the example given in the unittest section -above:: - - # models.py - - from django.db import models - - class Animal(models.Model): - """ - An animal that knows how to make noise - - # Create some animals - >>> lion = Animal.objects.create(name="lion", sound="roar") - >>> cat = Animal.objects.create(name="cat", sound="meow") - - # Make 'em speak - >>> lion.speak() - 'The lion says "roar"' - >>> cat.speak() - 'The cat says "meow"' - """ - name = models.CharField(max_length=20) - sound = models.CharField(max_length=20) - - def speak(self): - return 'The %s says "%s"' % (self.name, self.sound) - -When you :ref:`run your tests <running-tests>`, the test runner will find this -docstring, notice that portions of it look like an interactive Python session, -and execute those lines while checking that the results match. - -In the case of model tests, note that the test runner takes care of creating -its own test database. That is, any test that accesses a database -- by -creating and saving model instances, for example -- will not affect your -production database. However, the database is not refreshed between doctests, -so if your doctest requires a certain state you should consider flushing the -database or loading a fixture. (See the section on :ref:`fixtures -<topics-testing-fixtures>` for more on this.) Note that to use this feature, -the database user Django is connecting as must have ``CREATE DATABASE`` -rights. - -For more details about :mod:`doctest`, see the Python documentation. diff --git a/docs/topics/testing/index.txt b/docs/topics/testing/index.txt index 94e88bdf04..1a99a399b4 100644 --- a/docs/topics/testing/index.txt +++ b/docs/topics/testing/index.txt @@ -6,7 +6,6 @@ Testing in Django :hidden: overview - doctests advanced Automated testing is an extremely useful bug-killing tool for the modern @@ -29,83 +28,13 @@ it should be doing. The best part is, it's really easy. -Unit tests v. doctests -====================== - -There are two primary ways to write tests with Django, corresponding to the -two test frameworks that ship in the Python standard library. The two -frameworks are: - -* **Unit tests** -- tests that are expressed as methods on a Python class - that subclasses :class:`unittest.TestCase` or Django's customized - :class:`~django.test.TestCase`. For example:: - - import unittest - - class MyFuncTestCase(unittest.TestCase): - def testBasic(self): - a = ['larry', 'curly', 'moe'] - self.assertEqual(my_func(a, 0), 'larry') - self.assertEqual(my_func(a, 1), 'curly') - -* **Doctests** -- tests that are embedded in your functions' docstrings and - are written in a way that emulates a session of the Python interactive - interpreter. For example:: - - def my_func(a_list, idx): - """ - >>> a = ['larry', 'curly', 'moe'] - >>> my_func(a, 0) - 'larry' - >>> my_func(a, 1) - 'curly' - """ - return a_list[idx] - -Which should I use? -------------------- - -Because Django supports both of the standard Python test frameworks, it's up to -you and your tastes to decide which one to use. You can even decide to use -*both*. - -For developers new to testing, however, this choice can seem confusing. Here, -then, are a few key differences to help you decide which approach is right for -you: - -* If you've been using Python for a while, :mod:`doctest` will probably feel - more "pythonic". It's designed to make writing tests as easy as possible, - so it requires no overhead of writing classes or methods. You simply put - tests in docstrings. This has the added advantage of serving as - documentation (and correct documentation, at that!). However, while - doctests are good for some simple example code, they are not very good if - you want to produce either high quality, comprehensive tests or high - quality documentation. Test failures are often difficult to debug - as it can be unclear exactly why the test failed. Thus, doctests should - generally be avoided and used primarily for documentation examples only. - -* The :mod:`unittest` framework will probably feel very familiar to - developers coming from Java. :mod:`unittest` is inspired by Java's JUnit, - so you'll feel at home with this method if you've used JUnit or any test - framework inspired by JUnit. - -* If you need to write a bunch of tests that share similar code, then - you'll appreciate the :mod:`unittest` framework's organization around - classes and methods. This makes it easy to abstract common tasks into - common methods. The framework also supports explicit setup and/or cleanup - routines, which give you a high level of control over the environment - in which your test cases are run. - -* If you're writing tests for Django itself, you should use :mod:`unittest`. - Where to go from here ===================== -As unit tests are preferred in Django, we treat them in detail in the +The preferred way to write tests in Django is using the :mod:`unittest` module +built in to the Python standard library. This is covered in detail in the :doc:`overview` document. -:doc:`doctests` describes Django-specific features when using doctests. - -You can also use any *other* Python test framework, Django provides an API and +You can also use any *other* Python test framework; Django provides an API and tools for that kind of integration. They are described in the :ref:`other-testing-frameworks` section of :doc:`advanced`. diff --git a/docs/topics/testing/overview.txt b/docs/topics/testing/overview.txt index 9228a07b31..5023e099aa 100644 --- a/docs/topics/testing/overview.txt +++ b/docs/topics/testing/overview.txt @@ -17,7 +17,7 @@ Writing tests ============= Django's unit tests use a Python standard library module: :mod:`unittest`. This -module defines tests in class-based approach. +module defines tests using a class-based approach. .. admonition:: unittest2 @@ -46,16 +46,6 @@ module defines tests in class-based approach. .. _unittest2: http://pypi.python.org/pypi/unittest2 -For a given Django application, the test runner looks for unit tests in two -places: - -* The ``models.py`` file. The test runner looks for any subclass of - :class:`unittest.TestCase` in this module. - -* A file called ``tests.py`` in the application directory -- i.e., the - directory that holds ``models.py``. Again, the test runner looks for any - subclass of :class:`unittest.TestCase` in this module. - Here is an example :class:`unittest.TestCase` subclass:: from django.utils import unittest @@ -71,21 +61,18 @@ Here is an example :class:`unittest.TestCase` subclass:: self.assertEqual(self.lion.speak(), 'The lion says "roar"') self.assertEqual(self.cat.speak(), 'The cat says "meow"') -When you :ref:`run your tests <running-tests>`, the default behavior of the test -utility is to find all the test cases (that is, subclasses of -:class:`unittest.TestCase`) in ``models.py`` and ``tests.py``, automatically -build a test suite out of those test cases, and run that suite. +When you :ref:`run your tests <running-tests>`, the default behavior of the +test utility is to find all the test cases (that is, subclasses of +:class:`unittest.TestCase`) in any file whose name begins with ``test``, +automatically build a test suite out of those test cases, and run that suite. -There is a second way to define the test suite for a module: if you define a -function called ``suite()`` in either ``models.py`` or ``tests.py``, the -Django test runner will use that function to construct the test suite for that -module. This follows the `suggested organization`_ for unit tests. See the -Python documentation for more details on how to construct a complex test -suite. +.. versionchanged:: 1.6 -For more details about :mod:`unittest`, see the Python documentation. + Previously, Django's default test runner only discovered tests in + ``tests.py`` and ``models.py`` files within a Python package listed in + :setting:`INSTALLED_APPS`. -.. _suggested organization: http://docs.python.org/library/unittest.html#organizing-tests +For more details about :mod:`unittest`, see the Python documentation. .. warning:: @@ -101,6 +88,7 @@ For more details about :mod:`unittest`, see the Python documentation. .. _running-tests: + Running tests ============= @@ -109,46 +97,47 @@ your project's ``manage.py`` utility:: $ ./manage.py test -By default, this will run every test in every application in -:setting:`INSTALLED_APPS`. If you only want to run tests for a particular -application, add the application name to the command line. For example, if your -:setting:`INSTALLED_APPS` contains ``'myproject.polls'`` and -``'myproject.animals'``, you can run the ``myproject.animals`` unit tests alone -with this command:: +Test discovery is based on the unittest module's `built-in test discovery`. By +default, this will discover tests in any file named "test*.py" under the +current working directory. - $ ./manage.py test animals +.. _built-in test discovery: http://docs.python.org/2/library/unittest.html#test-discovery -Note that we used ``animals``, not ``myproject.animals``. +You can specify particular tests to run by supplying any number of "test +labels" to ``./manage.py test``. Each test label can be a full Python dotted +path to a package, module, ``TestCase`` subclass, or test method. For instance:: -You can be even *more* specific by naming an individual test case. To -run a single test case in an application (for example, the -``AnimalTestCase`` described in the "Writing unit tests" section), add -the name of the test case to the label on the command line:: + # Run all the tests in the animals.tests module + $ ./manage.py test animals.tests - $ ./manage.py test animals.AnimalTestCase + # Run all the tests found within the 'animals' package + $ ./manage.py test animals -And it gets even more granular than that! To run a *single* test -method inside a test case, add the name of the test method to the -label:: + # Run just one test case + $ ./manage.py test animals.tests.AnimalTestCase - $ ./manage.py test animals.AnimalTestCase.test_animals_can_speak + # Run just one test method + $ ./manage.py test animals.tests.AnimalTestCase.test_animals_can_speak -You can use the same rules if you're using doctests. Django will use the -test label as a path to the test method or class that you want to run. -If your ``models.py`` or ``tests.py`` has a function with a doctest, or -class with a class-level doctest, you can invoke that test by appending the -name of the test method or class to the label:: +You can also provide a path to a directory to discover tests below that +directory:: - $ ./manage.py test animals.classify + $ ./manage.py test animals/ -If you want to run the doctest for a specific method in a class, add the -name of the method to the label:: +You can specify a custom filename pattern match using the ``-p`` (or +``--pattern``) option, if your test files are named differently from the +``test*.py`` pattern:: - $ ./manage.py test animals.Classifier.run + $ ./manage.py test --pattern="tests_*.py" -If you're using a ``__test__`` dictionary to specify doctests for a -module, Django will use the label as a key in the ``__test__`` dictionary -for defined in ``models.py`` and ``tests.py``. +.. versionchanged:: 1.6 + + Previously, test labels were in the form ``applabel``, + ``applabel.TestCase``, or ``applabel.TestCase.test_method``, rather than + being true Python dotted paths, and tests could only be found within + ``tests.py`` or ``models.py`` files within a Python package listed in + :setting:`INSTALLED_APPS`. The ``--pattern`` option and file paths as test + labels are new in 1.6. If you press ``Ctrl-C`` while the tests are running, the test runner will wait for the currently running test to complete and then exit gracefully. @@ -173,6 +162,7 @@ be reported, and any test databases created by the run will not be destroyed. flag areas in your code that aren't strictly wrong but could benefit from a better implementation. + .. _the-test-database: The test database |