Version 3.0

Posted on Sun 28 November 2010 in Dispatches • 5 min read

After lots of hard work, some swearing, and lots of patience on the part of my wife, Version 3.0 of this site is now live. It’s been a long journey, but getting it done has been very satisfying.

A Little History

When I first switched to using my custom Django-based CMS, I used the opportunity as a means of learning both Django and Python, and I made a lot of novice mistakes. My code was riddled with hacky solutions to simple problems, but I was so happy just to have completed a working project I went ahead and launched it anyway. Even worse, due to some dependencies, I launched my site based on Django trunk, with the foolish belief that I would be able to keep up with changesets as they were published, a plan which fell apart almost immediately. In addition, there were gross mistakes in my code, especially in my custom comments application, which forced me to turn off comments completely.

With all of these issues in the existing site, it was time to update the code. Initially, I began working on version 2.0 of the site, and had a simple design plan.

  • Update existing code to be compatible with Django stable >= 1.0
  • Fix comments app

After a few weeks of work, I quickly realized that trying to simply update my novice code was going to create more headaches than it was worth. I consoled myself that I would just launch version 2.0 and then do a more thorough rewrite later, but then, life intervened. I got a new job and relocated for it, and I quickly found myself to be so busy that I didn’t even have much time to write posts on the site, much less any code.

About a year later, I came back and looked at version 2.0 and found myself really frustrated with the self-limiting foolishness of the codebase. In frustration, I started working on a new version based off of an empty project, the specs ambitious enough I felt like it deserved a whole new version number. The project was begun with the following design goals:

  • Be compatible with latest stable Django codebase
  • Denormalized data to ease the creation of lifestreaming-based views if necessary.
  • Custom content licensing per object
  • Privacy control settings for all content
  • All commenters as authenticated users
  • Allow authentication from OpenID, Facebook or Twitter
  • Use Django’s included comments app with custom moderators
  • Asynchronous moderation of comments
  • Flexible and extendable with few dependencies between applications

With the release of version 3.0, I am pleased to say that all of those goals have been met, including a few others I found along the way.

What’s New?

Version 3.0 is based upon an entire rewrite of the application code, and is designed to be flexible, efficient and easy to maintain. There are tons of new features, but most of them are on the backend. From the reader’s perspective, the changes are somewhat minimal, but I’d like to highlight a few of the interface changes I’m most pleased with.

Comments & Authentication

The issue that prompted the initial rewrite in the first place is finally resolved. Comments are now enabled on the site again. However, I no longer accept anonymous comments. Any reader wishing to leave a comment must authenticate through one of the following services:

Each user account created via these services receives a basic profile containing settings for how you would like your name to appear in the comments. I would prefer that you use you real name.

I’m using Celery and RabbitMQ to manage asyncronous moderation tasks such as running comments through Akismet, in addition to Django’s built-in spam protection technologies.

Search has been vastly improved in version 3.0, as I have replaced my ham-fisted search solution based on full-text search in PostrgreSQL with one utilizing Haystack with Xapian as the search backend. This allows me to provide very fast search functionality, with highlighted results and spelling suggestions. Eventually, I’ll be able to expand this to include faceted searches and advanced search operations.

In addition to the basic search functions, this approach has enabled me to provide “More Like This” functionality with each post on the site. At the bottom of each post, you will now find a listing of similar content on the site if you are interested in reading more. This content is also embedded in feed items as well. It looks a little something like this:

More like this

More Like This

I’m hoping that this, coupled with more reliable tag-based views should allow for better content discovery within the site’s archives.

Privacy Controls

I’m not currently using this new feature yet, but new privacy controls will allow me to create private content for friends and family in the future, that ties into some hooks in the user profiles for authenticated users.

Lessons Learned

It hasn’t been all wine and roses, however, and there have been lessons learned along the way. A few highlights worth pointing out.

Signals & save()

It can be really tempting to override save() methods of your models in Django. Don’t. Use the pre_save and post_save signals. That’s the reason they exist. There’s probably nothing technically wrong with overriding the save() method, but using signals will keep that logic separate and easy to enable/disable.

Use mod_wsgi In Daemon Mode

Seriously, you’ll read all over the place how using mod_wsgi’s embedded mode will get you better performance, but you better be an expert on tuning Apache in order to get it. Daemon mode makes it easy to reload your code base without restarting your webserver, and will use significantly less memory.

Virtualenv Is Your Friend

Virtualenv is not just your friend in this process, it is your best friend. The ability to set up isolated Python environments for your project is essential. It allows you to develop for specific versions of Python modules and easily replicate this environment between boxes. Learn it and use it. I found this tutorial particularly helpful.

Back to Work

Now that this revision is finally done, I can finally get back to the work of actually producing content for the site, so hopefully you should be seeing a lot more content coming here, and I apologize if you’ve had any duplicate entries appear in the feed readers during the transition.

Thanks again for your patience, and let me know what you think in the comments.