What people want out of Plone

Ever wondered what other people think about our favorite CMS Plone? I did, so yesterday during my talk at the annual Dutch Plone mini-conf I asked. Before reading the rest of the post think about what feature you like and one that you miss most regarding Plone.

At the beginning of my presentation I handed out some post-its and markers. Each person was to write one feature they like best and one feature they missed the most. By adding a + (liked) or – (missed) to each paper we could easily let the papers go round and still make them out.

Jan Murre, who also works at Pareto, helped me with entering the data into text files during my talk. He used a simple Python script to automatically sort the results.

Thought of your favorite and most missed feature yet? Then scroll past Jan to see the results.

280434001_dcddcb6b28

Votes Liked
5 Workflow
5 Flexibility
2 Community
2 Securitymodel
2 Standards
2 Extensible
2 Complete system out-of-the-box
1 Usability
1 Uniformity
1 Extensive
1 Python
1 Portlets
1 Manage and control branding
1 Folder structure
1 Open source
1 PAS (Plone’s authentication system)
1 In-site editing
1 Integration options
1 Security through using uncommon technology
1 Add-ons
Votes Missed
8 Performance
2 Kupu
2 Learning curve
1 Drag-drop support
1 Developer documentation
1 Bloat
1 Semantics
1 Database connectivity
1 Complete system out-of-the-box
1 Collections interface
1 Through the web editing of code etc.
1 Boilerplate (there is to much of it)
1 Agenda
1 No separate back-end for editing content
1 Flexibele layout
1 Categories
1 Collections
1 No automatic upgrade
1 Beer
1 Not enough themes
1 Heavy
1 Integration options
1 Layout for subsites

The audience consisted of both developers and actual end-users (though might have been a bit more technical people). After letting everyone know what the results where we started to discuss a few of these items.

Some of the negatives are already being worked (most people where looking forward to Plone 4 & 5 for performance improvements for instance). Others already have add-ons available (which could use some better marketing since not everyone knew about them).

The talk has been recorded so I might be able to give you more details at a later time on the specific discussions some of these points triggered. In the mean time please let me know what you thought of by adding a comment.

Advertisements

Exit code, colors, etc. when testing Plone

Do you want to have colored output showing problems with your tests? Are you looking to setup an automated system like Buildbot to run your tests? If so then this little tip could help you.

When you run your Plone tests you normally use zope.testing. The version shipped with Zope 2 that
you normally get is outdated. The newer versions have a lot of extra functionality. Fortunately it is pretty easy to get a newer version setup.

Edit your buildout.cfg and add versions section and a reference to it in you buildout section:

[buildout]
versions = versions

[versions]
zope.testing = 3.7.1

The last line tells buildout what version it needs to enforce. Now edit your instance section (the part with recipe = plone.recipe.zope2instance) and add zope.testing to the eggs = option.

Now enjoy your new version.

Stuff the catalog

Adding extra information to the portal catalog can be an easy way to get information from across your site. Plone let’s you easily create extra indexes using a generic setup profile. If you have an existing profile you can add a file catalog.xml with the following contents:

<?xml version=”1.0″?>
<object name=”portal_catalog” meta_type=”Plone Catalog Tool”>
<column value=”FirstLetter”/>
<index name=”FirstLetter” meta_type=”FieldIndex”>
<indexed_attr value=”FirstLetter”/>
</index>
</object>
This will create both a field index named FirstLetter and a corresponding meta data column (this will be available on the result objects when querying the catalog). There are many kind of indexes you can create this way. You can see how to use some of them in Plone’s own catalog.xml file.

Now we need to make sure we that our FirstLetter index will be filled. We can do this by creating a simple Python function. Let’s create a new file for this in our project named catalog.py with this content:

from Products.CMFPlone.CatalogTool import registerIndexableAttribute

def first_letter(obj, portal, vars, **kwargs):
     # Make sure we have a title with something in it
    if obj.Title():
        # Return the first letter
        return obj.Title()[0].lower()

registerIndexableAttribute('FirstLetter', first_letter)

This will register an extra handler for Plone’s indexing system. To make sure it is registered we need to import the catalog module from the __init__.py of the project/product. Just add a line like this to your project (replace exampleproject with your project’s name):

import exampleproject.catalog

This should make the FirstName index available when the project is installed. Now you query your catalog against this with something like:

portal_catalog(FirstLetter='a')

Hiding your dependencies

A few days ago Wichert showed me something interesting. You can hide profiles / products from both the quickinstaller and the Plone site creation screen. This came in handy for a project we were working on.

The project is for a client that delivers a customized version of a product we developed for them. So each of their clients receives a tailored version (think branding, default settings etc.).

Code wise this means we have a base package for all installaties. On top of this we built the branded versions. To minimize installation problems the possibility to hide dependent products came in really handy.

Hiding products can be done by registering two utilities. One for disabling products in the quickinstaller. The other for disabling extension profiles from the Plone site creation.

To register these utilities add something like this your configure.zcml:

<utility factory=".profile.HiddenProducts" name="my.special.project" />
<utility factory=".profile.HiddenProfiles" name="my.special.project" />

Now create a profile.py file in your project and add the following lines:

from zope.interface import implements
from Products.CMFQuickInstallerTool.interfaces import INonInstallable as INonInstallableProducts
from Products.CMFPlone.interfaces import INonInstallable as INonInstallableProfiles

class HiddenProducts(object):
implements(INonInstallableProducts)

def getNonInstallableProducts(self):
return [
u'some.package',
u'another.product',
]

class HiddenProfiles(object):
implements(INonInstallableProfiles)

def getNonInstallableProfiles(self):
return [
u'some.package:default',
u'another.product:default',
]

Open Python eggs directly from Emacs

Emacs can open compressed files. Since Python eggs are just compressed ZIP files we can tell Emacs how to open them. Add the following to your .emacs file.

;; Use archive mode to open Python eggs
(add-to-list 'auto-mode-alist '("\\.egg\\'" . archive-mode))

Now you can open .egg files and get something like:

Speed up your functional Django tests

Test should run as fast as possible to really integrate them into your development flow. You can imagine my annoyance when our project with 100+ functional tests took half a minute to run them all. We where already using SQLite’s in memory storage for our tests so that shouldn’t be the problem.

Up until that point we had been using the default Django unittest test case which uses: call_command('flush', verbosity=0, interactive=False). After some profileing I came to the conclusion that most of the time was spend in this call.

Luckily I rememberd a recent post about using transactions for clearing the database state. After changing our tests to use transaction instead of flushing they became a lot quicker. They now run in less than 7 seconds (was 30+ seconds).

The code you see below can be used as a simple drop in replacement for the build-in Django test case. Most of the code has been taken from the normal Django test case, only the transactions are new.

from django.db import transaction
from django.core import mail
from django.test import TestCase as DjangoTestCase
from django.core.management import call_command
from django.conf import settings
from django.core.urlresolvers import clear_url_caches

class TestCase(DjangoTestCase):

    def _pre_setup(self):
        transaction.enter_transaction_management()
        transaction.managed(True)
        if hasattr(self, 'fixtures'):
            # We have to use this slightly awkward syntax due to the fact
            # that we're using *args and **kwargs together.
            call_command('loaddata', *self.fixtures, **{'verbosity': 0})
        if hasattr(self, 'urls'):
            self._old_root_urlconf = settings.ROOT_URLCONF
            settings.ROOT_URLCONF = self.urls
            clear_url_caches()
        mail.outbox = []

    def _post_teardown(self):
        transaction.rollback()
        super(TestCase, self)._post_teardown()

Repeatable Django installations

Don’t you hate it when you have to spend lots of time tracking down dependencies etc. when joining an existing a project? Did you know that it doesn’t have to be this way?

Clever folks which where in the same boat as us have created zc.buildout. This tool allows you to easily configure all your software requirements with a simple text file. The example below can be used to install a fully working Django trunk version with ipython and satchmo included.

[buildout]
parts = satchmo django
eggs = ipython

[satchmo]
recipe = gocept.download
url = http://www.satchmoproject.com/snapshots/satchmo-0.6.tar.gz
md5sum = 659a4845c1c731be5cfe29bfcc5d14b1

[django]
recipe = djangorecipe
version = trunk
settings = development
eggs = ${buildout:eggs}
pythonpath =
  ${satchmo:location}
project = dummyshop

This example shows the basic structure of a buildout file. You can turn this into a working environment by adding a bootstrap.py file. After that you can run;

python bootstrap.py

bin/buildout -v

There should now be a bin/django script which does all the things that manage.py would normally do.

The buildout configuration itself consists mainly of a few sections (marked with […]). Each section specifies a recipe to be used. Buildout basically uses these recipes to build django etc. To see what you can do with a recipe you can look-up it’s documentation. In the case of django you can find it at djangorecipe’s documentation page.

There are many more recipe’s at the package index to do all sorts of interesting things. So please do yourself a favour and give buildout a spin.

buildout.el, Emacs bliss for buildouts

Emacs can be a nice OS for doing Python development with. You can use the built-in eshell to execute your code, start your server and run your tests. It is even nicer to use things like gud, compile mode and tags. If you are working on multiple projects (like I am) over the course of a day it can be a bit to much to M-x pdb <enter full path to bin/instance> every time.

This is where buildout.el comes in. It has a few interactive functions (which you should bind to a shortcut, like the F1-12 keys) to execute these things. It knows where you are by checking the location of the current buffer and then crawls up until it finds something which looks like a buildout directory.

These are the features it currently provides:

  • Buildout invocation (with bootstrap if necessary)
  • Creation of a tags file (using etags)
  • Running the tests in compile mode (colors + jump to the line which triggered the failure)
  • Start your app server (bin/instance or bin/django) in a gud session

If you have any suggestions for improvements please let me know so we can make it even better. The only thing left to do is to download buildout.el, just add something like (load-library “buildout”)
to your .emacs and add keybindings. These are mine:

;; Key bindings
(global-set-key ‘[f5] ‘run-application-based-on-buffer)
(global-set-key ‘[f9] ‘run-buildout-based-on-buffer)
(global-set-key ‘[f8] ‘run-etags-based-on-buffer)
(global-set-key ‘[f12] ‘run-tests-based-on-buffer)

KSS-mode

Previously KSS only had editor support in Vim. This made the kitchen sink of editing a bit jealous. Fortunately we now have a Emacs mode for KSS.

Screenshot of KSS-mode in action

I had one problem with getting the highlighting for CSS selectors to work with complex ones (see the screenshot). If there are any Emacs wizards out there willing to help that would be greatly appreciated.

KSS sprint has started

img_5675.jpgWe started with the KSS sprint today. One of the first improvements is that base2’s CSS selector is now usable with KSS. This means that all applications using KSS will get a nice speedup (base2 should be a lot faster than the original CSSQuery).

The fix for this entailed change the protocol between the client and the server. In the previous version the data communicated would be structured XML. This caused problems because the DOM extracted from this contained namespaces (which broke XPath).

img_5676.jpgOur new protocol transfers data as string’s (either within a CDATA or in a text node). As a side effect of this the DOM injection/replacement actions had to be changed. Because the code previously had to deal with either DOM or string data we could clean it up quite a bit. This means a leaner and meaner KSS.

A thing I worked on by myself was merging the pure Python packages for KSS into one package. This new package will probably become the base for a new kss.zope package which will (with BBB) replace kss.core. The kss.pylons pacakge is already changed so that it now depends on kss.base instead of the separate packages.