Archive for the ‘django’ Category

Speed up your functional Django tests

Test should run as fast as possible to really integrate them into your development flow. You can imagine my annoyance when our project with 100+ functional tests took half a minute to run them all. We where already using SQLite’s in memory storage for our tests so that shouldn’t be the problem.

Up until that point we had been using the default Django unittest test case which uses: call_command('flush', verbosity=0, interactive=False). After some profileing I came to the conclusion that most of the time was spend in this call.

Luckily I rememberd a recent post about using transactions for clearing the database state. After changing our tests to use transaction instead of flushing they became a lot quicker. They now run in less than 7 seconds (was 30+ seconds).

The code you see below can be used as a simple drop in replacement for the build-in Django test case. Most of the code has been taken from the normal Django test case, only the transactions are new.

from django.db import transaction
from django.core import mail
from django.test import TestCase as DjangoTestCase
from django.core.management import call_command
from django.conf import settings
from django.core.urlresolvers import clear_url_caches

class TestCase(DjangoTestCase):

    def _pre_setup(self):
        transaction.enter_transaction_management()
        transaction.managed(True)
        if hasattr(self, 'fixtures'):
            # We have to use this slightly awkward syntax due to the fact
            # that we're using *args and **kwargs together.
            call_command('loaddata', *self.fixtures, **{'verbosity': 0})
        if hasattr(self, 'urls'):
            self._old_root_urlconf = settings.ROOT_URLCONF
            settings.ROOT_URLCONF = self.urls
            clear_url_caches()
        mail.outbox = []

    def _post_teardown(self):
        transaction.rollback()
        super(TestCase, self)._post_teardown()
Advertisements

Repeatable Django installations

Don’t you hate it when you have to spend lots of time tracking down dependencies etc. when joining an existing a project? Did you know that it doesn’t have to be this way?

Clever folks which where in the same boat as us have created zc.buildout. This tool allows you to easily configure all your software requirements with a simple text file. The example below can be used to install a fully working Django trunk version with ipython and satchmo included.

[buildout]
parts = satchmo django
eggs = ipython

[satchmo]
recipe = gocept.download
url = http://www.satchmoproject.com/snapshots/satchmo-0.6.tar.gz
md5sum = 659a4845c1c731be5cfe29bfcc5d14b1

[django]
recipe = djangorecipe
version = trunk
settings = development
eggs = ${buildout:eggs}
pythonpath =
  ${satchmo:location}
project = dummyshop

This example shows the basic structure of a buildout file. You can turn this into a working environment by adding a bootstrap.py file. After that you can run;

python bootstrap.py

bin/buildout -v

There should now be a bin/django script which does all the things that manage.py would normally do.

The buildout configuration itself consists mainly of a few sections (marked with […]). Each section┬áspecifies a recipe to be used. Buildout basically uses these recipes to build django etc. To see what you can do with a recipe you can look-up it’s documentation. In the case of django you can find it at djangorecipe’s documentation page.

There are many more recipe’s at the package index to do all sorts of interesting things. So please do yourself a favour and give buildout a spin.

buildout.el, Emacs bliss for buildouts

Emacs can be a nice OS for doing Python development with. You can use the built-in eshell to execute your code, start your server and run your tests. It is even nicer to use things like gud, compile mode and tags. If you are working on multiple projects (like I am) over the course of a day it can be a bit to much to M-x pdb <enter full path to bin/instance> every time.

This is where buildout.el comes in. It has a few interactive functions (which you should bind to a shortcut, like the F1-12 keys) to execute these things. It knows where you are by checking the location of the current buffer and then crawls up until it finds something which looks like a buildout directory.

These are the features it currently provides:

  • Buildout invocation (with bootstrap if necessary)
  • Creation of a tags file (using etags)
  • Running the tests in compile mode (colors + jump to the line which triggered the failure)
  • Start your app server (bin/instance or bin/django) in a gud session

If you have any suggestions for improvements please let me know so we can make it even better. The only thing left to do is to download buildout.el, just add something like (load-library “buildout”)
to your .emacs and add keybindings. These are mine:

;; Key bindings
(global-set-key ‘[f5] ‘run-application-based-on-buffer)
(global-set-key ‘[f9] ‘run-buildout-based-on-buffer)
(global-set-key ‘[f8] ‘run-etags-based-on-buffer)
(global-set-key ‘[f12] ‘run-tests-based-on-buffer)