Django Software Stack

Here at Sourcefabric we have been looking in to what software stack to use for our next web products. This discussion used to be (oh, 5-10 years ago) a lot simpler, mainly involving choosing an operating systems, a language, and a database. But now it’s about higher-level things like out-of-the-box support for commenting, activity streams, ajax, cloud storage, deployment tools, scalability, geolocation, tagging, RESTful interfaces, search, twitter, facebook, and a whole lot more. Users and customers are starting to expect all of this built into any web application you build. Building applications these days is largely about integrating a large number of existing technologies instead of building something from scratch. It’s more like we are becoming our own little version of RedHat, Novell, or Canonical: taking a bunch of stuff that already exists, put it together, test that it all works, fix the bugs, add your 1% of special code, and ship it. As far as (non-mission critical)web frameworks go, I think these days you have two options: Python/Django and Ruby/Rails.

We had a brief moment where some of us were considering building on top of Drupal, but a fantastic article on why that isn’t a good idea(for us) can be found here: “Drupal or Django? A Guide for Decision Makers“. If you don’t want to read the whole thing, here is the summary: Django(a framework) gives you flexibility to evolve, while Drupal(a CMS) gives you the basic stuff faster. Django allows you to put the pieces together in any way you want, but a developer has to put it together. Drupal gives you plugins that work right away, but changing how they work can be difficult and time consuming. And there is also the very large bonus of using Python for Django versus PHP for Drupal. I also suggest reading these comments to that article: one, two, three, four.

As far as Python versus Ruby, there aren’t vast differences functionality-wise in these languages. But if you look at the ecosystem around Python, you will generally find that it’s more mature than Ruby at the moment. The libraries tend to be more mature, especially for audio processing which is important for us. If you look at the speed of execution of these languages, Ruby seems to have recently almost caught up with Python. However, there are folks(including the inventor of Python, Guido VR), backed by Google, working on something called “Unladen Swallow” which is a a new Python interpreter with the potential to speed up Python by a factor of five(!). Then there is the fact the Google supports Python and Django on it’s app engine. Finally, and probably unsurprising to everyone reading this, all the developers here already know Python to some degree. So currently we are planning to go with Python and Django.

There doesn’t seem to be any place on the web which is something like a “Guide to the Django Software Stack”, so we put together a high-level quick and dirty one (below). If anyone knows of a guide like that, please post a link to it in the comments! Note that the guide below is simply from doing research and not actually using anything yet. Once we get the development going, we’ll update you on the stack we actually end up using. Also note that this is only a list of Django technology and not Python tech; to include all the Python tech would make this article too long.

Django

First there is Django itself, and out of the box it comes with:

Pinax

On top of that there is a project called Pinax, which is a set of plugins (called “apps” in Django) which run on Django. Currently there are apps for:

• OpenID support
• Email verification
• Password management
• Site announcements
• Notification framework
• User-to-user messaging
• Friend invitation (both internal and external to the site)
• Basic twitter clone
• OEMBED support
• Gravatar support
• Interest groups (called tribes)
• Projects with basic task and issue management
• Threaded discussions
• Wikis with multiple markup support
• Blogging
• Bookmarks
• Tagging
• Contact import (from vCard, Google or Yahoo)
• Photo management
• Static Files

REST Interface

Everyone needs a RESTful interface these days, so the standard for Django seems to be:

• Piston – It’s not magic though, you still need some coding for each model to hook it in: Overview and Tutorial
• Django-ROA – supposedly this is more like magic, automatic RESTful API built on top of Piston, something like what Ruby does.

Standard Plumbing

I would think that many apps could use the following:

• Haystack, search for Django (also: Sphinx)
• Arecibo, automated centralized error logging – i.e. an error in any installation will be sent to a central database. Has a nice web interface for viewing the bugs.
• Jinja, a templating system that provides more functionality than default Django
• South, a database migration tool specifically for Django
• Fabric, a Python deployment app to push your app to many servers at once
• Celery, distributed message passing(can be used for distributed tasks)
• Virtualenv, a tool to create isolated Python environments so that upgrading the software for one python web site doesnt break the other sites (built into Pinax)
• Paste, a middleware layer that enables you to debug Python web apps
• Django-DBSettings: Application settings whose values can be updated while a project is up and running
• Django-LiveSettings: provides the ability to configure settings via an admin interface, rather than by editing “settings.py”.
• Django-Config: A simple way to manage profile based settings in django
• Translation (Article)
  • Rosetta: eases the translation process of your Django projects.
  • Django-Datatrans: Translate Django models without changing anything to existing applications and their underlying database.
• Article: “Making development with Django more fun with Behavior-Driven Development under Freshen”
  • Freshen, a clone of Cucumber, a Behaviour Driven Development tool
  • Nose, easier unit testing
  • Mock, a tool for creating mock api’s for testing purposes
• FileTransfers
• Sore Thumb, image processing for thumbnails (Article)

Python Package Index (PyPI)

And finally there is PyPI, or “Python Package Index”, a repository of Python libraries. It’s kind of like apt-get for you Ubuntu/Debian people, or the App Store for you iPhone people. You can download and install a package like so: “easy_install SQLObject” from the command line. (Docs are here)

Click Django apps on PyPI to see all Django-related apps on PyPI. Note however that many of these packages seem to be very early-stage code (version numbers < ; 1.0). In the examples below, I generally tried to find packages that were more mature but it wasn’t always possible. Examples:

• Plugin Framework, create plugin-points in templates & manage plugins (Note there seem to be many plugin frameworks listed, this is just one of them.)
• TinyMCE Plugin / WYSWIYG Plugin
• Django Permissions
• Django DB_Dump, backup and restore django DBs
• Workflows
• Job scheduling
• Monitor ORM queries
• Hierarchical data, also another lib here
• Test coverage
• Dojo serializer
• Activity Stream

Community

What about the Django community? Here are some web sites for developers:

• Django Snippets, little bits of code for getting small tasks done in Django (i.e. any code that’s useful but isn’t an app)
• Planet Django, a blog with Django news
• Django Hosting, hosting ISPs for DJango

To all of the more experienced Django people out there: what other libraries or apps would you consider essential for development? How about community sites? Please post a comment and I will update the post!

Another good article based on something actually implemented: The Apps That Power Django-Mingus

Sourcefabric is looking for Django developers for our Prague and Toronto offices. To apply, send your cover letter and CV to: paul dot baranowski at sourcefabric dot org and mugur dot rus at sourcefabric dot org