Tim Child

As part of my on going effort for realtime web updates on one of our applications I needed to install RabbitMQ HEAD (That is the development head) with the RabbitMQ STOMP adaptor. Talking on IRC channel it was recommended to me to install from source, but unfortunately the documentation is out of date. So the following will do it cleanly:

Updated: The following will create a Debian style package and install (replace ** with the version numbering that you wish to use:

Activating the STOMP plugin:

Configuration file:

Restart to activate (the activation script for plugins is depreciated):

And you should have a working RabbitMQ on Ubuntu with a STOMP adaptor.

Today I got my Django project working with Orbited.

By creating a twisted proxy for Orbited and Django I am able to serve a development environment for both Django and Orbited plus the builtin MorbidQ message queue.

I am going to be using this to give live notifications of events that are happening on the system in realtime, and so it should be able to handle quite a large load. To do this I will use another queue in production, but I have found it needed to sort out the development environment first of course.

This daemon I have put up at GitHub .

It even serves the static files. Much thanks goes to the HotDot project for seeing how alot of this was done, but I have taken out the authentication and other bits to make it cleaner for general use.

If you are having problems with _PQbackendPID, psycopg2 and OS X and have tried the forcing to 32 bit mentioned in other posts about the web then it might be time to try a more forceful approach. This is an example of the error that you might face:

The first thing to do is get rid of your old Psycopg2 (just in case), so locate the site-packages directory:

Then in that directory:

Then remove the old version of PostGreSQL as much as possible. Not sure what version you might have installed, but if you have pg_config on your sys path it will tell you where it is. These instructions may or may not work for you:

1. Stop the server. sudo launchctl unload /Library/LaunchDaemons/com.edb.launchd.postgresql-8.4.plist
2. Remove the plist file
   sudo rm -f /Library/LaunchDaemons/com.edb.launchd.postgresql-8.4.plist
3. Remove the Applications Menu.
   sudo rm -f “/Applications/PostgreSQL 8.4″
4. Remove the installation directory
   sudo rm -rf /Library/PostgreSQL/8.4
   (Note: You can backup your data directory in case you need it.)
5. Remove the ‘postgres’ user
   sudo dscl . delete /Users/postgres
   (Note: This step is optional)
6. Remove the ini file
   sudo rm -f /etc/postgres-reg.ini

I found a reboot worked to make sure that we are back to normal and nothing is installed.

Download psycopg2-2.2.1.tar.gz
Download PostGreSQL from http://www.enterprisedb.com/products/download.do – the version that isn’t the “Standard Server” but just “PostgreSQL 8.4 ” – you do have to register for that.

Install Postgresql – making sure that it completes, if not it means that you probably had some old files or users in there. If it went ok it should have installed and started running on port 5432. Try removing it /Library/PostgreSQL/8.4/uninstall-postgresql.app – and then removing the user or files that it complained about.

Download and install http://www.python.org/download/releases/2.6.5/ Mac Installer disk image. Restart (not sure that this restart is totally needed, but I did it anyway).

Now when you are on a new terminal window and type “python” it should say version 2.6.5. You will have to reinstall any python bits and pieces that you have installed previously – but make sure that you reinstall PIP and easy_install – they will use the old version of Python if you are not careful.

Now go into the directory of psycopg2-2.2.1 that you downloaded earlier.

Change setup.cfg so it looks like the following:

Now run the installation using the new python:

This should install. You should be able to do “import psycopg2″ from the python prompt. Now install all the libraries and python modules that you need, and create the Postgresql database that you want (pgadmin3 is the best tool for this).

Please note this was done on a OS X Snow Leopard 10.6.4 with XCode and the 10.4 libraries installed.

My latest task and one that I have been researching and playing with for some time is to integrate real time feedback in to our web application. The main goal is to be able to provide real time feedback of jobs and updates to users without Ajax style polling.

This is a simple requirement to write, but one that had many deeper issues. Firstly it needs to be able to handle many users concurrently and secondly it should do so without blocking execution of code. Our application framework was not designed for this and this is no problem, it has been designed to do certain thing well and other I am really not going to force upon it.

The main application framework will handle the standard page requests, user authentication and the rest of the standard stack, so I have been looking at technologies to complement its capabilities. This has led me to look at Comet applications and very fast non-block frameworks.

Of these I would prefer to keep the language in Python, there seem to be good alternatives in Java and .Net but I would rather keep an upgrade path by having a language that can be migrated across to these (thanks Jython and IronPython) and keep data in a backend store that is language agnostic.

Orbited and Tornado are really good candidates. I am going to stay away from more raw implementations of non-blocking event based frameworks such as Twisted because efforts are often better at this stage in building to serve a business purpose where possible. Orbited does actually use twisted, and I am hoping that I will have little need to dig that deep except for understanding its implementation.

Tornado I initially didn’t think of, but came around after reading “Building the Realtime User Experience” in which the author, Ted Roden, seems rather partial to it. In fact it can work quite well in fronting our application, but I am starting to reconsider based upon two factors. Firstly, even though it itself was built for a largish scale operation (FriendFeed – now owned by FaceBook), it has little other commercial users behind it and I haven’t found many examples of usage with integrating with Django, Pylons, Turbogears or Zope in a production environment. Secondly what really turns me off is the immaturity of its tests. Looking on GitHub even the README shows that little effort has been put into its testing environment, let alone their sporadic coverage.

Twisted on the other hand is extremely mature in both terms of numbers of deployments and its test environment, and this makes Orbited a much better candidate.

I have yet to look at certain other alternatives, such as Diesel, but I have a feeling that I will again come around to start using Orbited.

The other huge plus for me in using Orbited is that it can use the STOMP protocol, and use RabbitMQ or ActiveMQ. Other parts of our technology stack are starting to use STOMP and I wish to move more operations on to an event queue, which will open up much in terms of flexibility in the future.

Its a pity that “Building the Realtime User Experience” was so limited in terms of covering alternatives to Tornado.

Wake-on-lan is incredibly useful for those scenarios where you have more than one machine on a network, but you don’t always want to have them running but they are needed sometimes.

To setup, make sure that the Motherboard and NIC support is there from your manufacturer. Sometimes it has to be enabled, such as making sure that it uses S3 for shutdown, and that PCI wake is supported.

Next you need to check support from the operating system, I am using Ubuntu 10.04, but this should work for other operating systems. Install ethtool, and then check against the ethernet adapter (eth0 in my example).

If support is enabled, but it is turned off you will see:

So we turn it on:

Now we need to make sure that its always enabled everytime we restart:

And in that file:

Now make sure that its put into the correct runlevels:

And that is it. It should be supported that you can wake the machine from another computer/server using your favourite wakeup utility ( such as wakeonlan ).