Tim Child

I recently had a problem with having duplicate MAC addresses on my network for a couple of machines that I was using in a Linux KVM cluster. Flashing the firmware on the motherboards seem to have reset the MAC address to a default (thanks ASUS) that was the same on each board.

Changing the MAC address back seems like quite a hard job and looking around on the internet it seems that many people are doing this the hard way with shell scripts and so forth. Instead its easy enough to do this using the scripts that are already setting up the network for Debian (and I am sure this will work for Ubuntu just as well).

Edit /etc/network/interfaces like so:

[cc lang="bash"]
auto eth0
iface eth0 inet manual
pre-up ifconfig $IFACE hw ether 00:01:02:03:04:05
pre-up ifconfig $IFACE up
pre-down ifconfig $IFACE down
[/cc]

What this is doing is manually controlling the startup and shutdown of the eth0 interface. I am using bridging on my KVM boxes, so I don’t actually have to set up the IP address (albeit DHCP or manual IP configuration), but that aside the interesting parts are what happens in pre-up. Firstly I set the hardware address to a new address (and please change it from the example above – thats not the actual one I used either), then we tell it to bring the interface up.

We also use pre-down to tell it what happens when stopping the network service.

Probably better than messing around with Shell scripts.

Really nice little round up on Quora:

http://www.quora.com/Linux/What-are-some-time-saving-tips-that-every-Linux-user-should-know

With a follow up on hacker news:

http://news.ycombinator.com/item?id=2361978

Probably going to be more of these.

This is probably the best over article on the whole WebM/VP8 and H.264 issue:

http://antimatter15.com/wp/2011/01/the-ambiguity-of-open-and-vp8-vs-h-264/

Well worth a read.

Django has a really nice select multiple field, for choosing multiple items at once with a Javascript chooser..

It is really handy to use when there is a large number of items that have to be chosen from in a select. I am using with JQuery.

To do this, grab SelectFilter2.js from the django-admin media folder, or get it from Django’s GitHub repository. Then to initialize it you can put the following into your code:

[cc lang="Javascript"]
$(document).ready(function() {
$.each($(“select[multiple]“), function () {
// “Locations” can be any label you want
SelectFilter.init(this.id, “Locations”, 0, “/media/”);
});
});
[/cc]

This will use the SelectFilter on all select multiples and give it a label of locations.

For reference the functions are:
[cc lang="javascript"]
function(field_id, field_name, is_stacked, admin_media_prefix)
[/cc]

So you could use:
[cc lang="Javascript"]
SelectFilter.init(“#MySelect”, “Test”, 0, “{{ MEDIA_URL}}img/”);
[/cc]

Then all that needs is styling the select multiple.

We have been using the localization and internationalization ( l10n& i18n ) from Django quite a bit for our translations, and although we haven’t had to tackle anything difficult like Arabic or Persian yet there are still some shortcomings in the system.

The standard Django module for input of dates and datetime stamps is quite restrictive. It depends on a module, I think written by Simon Willson which uses the PHP notation rather than the Python/C strftime / strptime notation. Thus although its easy to display dates or datetime based upon the included locale files, parsing them is not so trivial from what I can see. To get around it, a configuration is placed to try and parse the date or datetime’s until a match is found. Looking at the code it does indeed use a try and except method of matching, mean potentially 10/10/10 could be any combination.

Secondly we use the JS18n/ url which outputs translations to be used by Javascript, but this is not in the form that the Javascript Date object can understand so one has to create a function to parse the datetime:

[cc lang="Javascript"]
formats['DATETIME_FORMAT'] = ‘N j, Y, P’;
[/cc]

That can’t be understood by Javascript unless you do so.

Looks like there is some work to be done there, but mid-project is not the time to start making patches to code libraries.

Been using RabbitMQ quite a lot at Cantemo, where we use it for long standing jobs, working with other languages and for notifications.

But I needed to decouple some of the code from my application, to be able to distribute it and talk from other systems – exactly what we needed to do, run tasks that are in the application from another application via posting messages to RabbitMQ. Tasks are held in Celery, and started usually within the code base.

Staying with Python, though not always using CPython, I have a task “notification_of_upload_handler” which is registered in Celery (it actually has a full path to namespace it correctly, but there is no need to show that for my example).

Obviously to do this you will need, Celery (which includes carrot), and RabbitMQ with a user and virtual host.

Using the Carrot library, I can start this task by sending a message to RabbitMQ like so:

[cc lang="python"]
from carrot.connection import BrokerConnection
from carrot.messaging import Publisher
from celery.utils import gen_unique_id
conn = BrokerConnection(hostname=”rabbithost”, port=5672, userid=”myrabbituser”, password=”mypassword”, virtual_host=”rabbitvhost”)
publisher = Publisher(connection=conn, exchange=”celery”, routing_key=”celery”)
publisher.send({“task”:”notification_of_upload_handler”, “args”: (“more”,), “kwargs”: {}, “id”: gen_unique_id()})
publisher.close()
[/cc]

Of course you could use any language and any AQMP library to do this. Check out http://celeryq.org/docs/internals/protocol.html for more on the protocol.

My next task is to broadcast this so that messages don’t stay around.

I needed to test sending growl messages to machines running Growl, using just Python.

There was a problem trying to find scripts, old websites were not responding, and there was a distinct lack of being able to get anything off the group easily but I chanced upon a script that worked. Fearing that the site it is hosted on might also end up biting dust, I made my changes to the script and have uploaded it to GitHub:

http://github.com/timc3/netgrowl

So if you need to send Growl messages either from the command line or from within your Python project check it out.

The original authors are detailed in the script.

As part of my on going effort for realtime web updates on one of our applications I needed to install RabbitMQ HEAD (That is the development head) with the RabbitMQ STOMP adaptor. Talking on IRC channel it was recommended to me to install from source, but unfortunately the documentation is out of date. So the following will do it cleanly:

[cc lang="bash"]
sudo apt-get install erlang-crypto erlang-snmp erlang-syntax-tools libsctp1 lksctp-tools erlang-runtime-tools erlang-mnesia erlang-public-key erlang-os-mon erlang-ssl erlang-base erlang-parsetools mercurial git-core build-essential erlang-dev zip erlang-tools erlang-src python-simplejson erlang-edoc
cd ~
hg clone http://hg.rabbitmq.com/rabbitmq-public-umbrella
cd rabbitmq-public-umbrella
make co
make
[/cc]

Updated: The following will create a Debian style package and install (replace ** with the version numbering that you wish to use:
[cc lang="bash"]
sudo apt-get install cdbs debhelper xmlto
cd rabbitmq-server
make VERSION=1.8.** srcdist
make -C packaging/debs/Debian UNOFFICIAL_RELEASE=true package
sudo dpkg -i packaging/debs/Debian/rabbitmq-server_1.8.**-1_all.deb
[/cc]

Activating the STOMP plugin:
[cc lang="bash"]
cd rabbitmq-public-umbrella/rabbitmq-stomp/dist
sudo cp amqp_client.ez /usr/lib/rabbitmq/lib/rabbitmq_server-1.8.1/plugins
sudo cp rabbit_stomp.ez /usr/lib/rabbitmq/lib/rabbitmq_server-1.8.1/plugins
[/cc]

Configuration file:
[cc lang="bash"]
RABBITMQ_PLUGINS_DIR=/usr/lib/rabbitmq/lib/rabbitmq_server-1.8.1/plugins
SERVER_START_ARGS=’-rabbit_stomp listeners [{"0.0.0.0",61613}]‘
[/cc]

Restart to activate (the activation script for plugins is depreciated):
[cc lang="bash"]
sudo /etc/init.d/rabbitmq-server restart
[/cc]

And you should have a working RabbitMQ on Ubuntu with a STOMP adaptor.

Today I got my Django project working with Orbited.

By creating a twisted proxy for Orbited and Django I am able to serve a development environment for both Django and Orbited plus the builtin MorbidQ message queue.

I am going to be using this to give live notifications of events that are happening on the system in realtime, and so it should be able to handle quite a large load. To do this I will use another queue in production, but I have found it needed to sort out the development environment first of course.

This daemon I have put up at GitHub .

It even serves the static files. Much thanks goes to the HotDot project for seeing how alot of this was done, but I have taken out the authentication and other bits to make it cleaner for general use.

If you are having problems with _PQbackendPID, psycopg2 and OS X and have tried the forcing to 32 bit mentioned in other posts about the web then it might be time to try a more forceful approach. This is an example of the error that you might face:

[cc lang="bash"]
dlopen(/Library/Python/2.6/site-packages/psycopg2/_psycopg.so, 2): Symbol not found: _PQbackendPID
Referenced from: /Library/Python/2.6/site-packages/psycopg2/_psycopg.so
Expected in: flat namespace
in /Library/Python/2.6/site-packages/psycopg2/_psycopg.so
[/cc]

The first thing to do is get rid of your old Psycopg2 (just in case), so locate the site-packages directory:

[cc lang="bash"]python -c “from distutils.sysconfig import get_python_lib; print get_python_lib()”[/cc]

Then in that directory:
[cc lang="bash"]
rm -rf psycopg*
[/cc]

Then remove the old version of PostGreSQL as much as possible. Not sure what version you might have installed, but if you have pg_config on your sys path it will tell you where it is. These instructions may or may not work for you:

1. Stop the server. sudo launchctl unload /Library/LaunchDaemons/com.edb.launchd.postgresql-8.4.plist
2. Remove the plist file
   sudo rm -f /Library/LaunchDaemons/com.edb.launchd.postgresql-8.4.plist
3. Remove the Applications Menu.
   sudo rm -f “/Applications/PostgreSQL 8.4″
4. Remove the installation directory
   sudo rm -rf /Library/PostgreSQL/8.4
   (Note: You can backup your data directory in case you need it.)
5. Remove the ‘postgres’ user
   sudo dscl . delete /Users/postgres
   (Note: This step is optional)
6. Remove the ini file
   sudo rm -f /etc/postgres-reg.ini

I found a reboot worked to make sure that we are back to normal and nothing is installed.

Download psycopg2-2.2.1.tar.gz
Download PostGreSQL from http://www.enterprisedb.com/products/download.do – the version that isn’t the “Standard Server” but just “PostgreSQL 8.4 ” – you do have to register for that.

Install Postgresql – making sure that it completes, if not it means that you probably had some old files or users in there. If it went ok it should have installed and started running on port 5432. Try removing it /Library/PostgreSQL/8.4/uninstall-postgresql.app – and then removing the user or files that it complained about.

Download and install http://www.python.org/download/releases/2.6.5/ Mac Installer disk image. Restart (not sure that this restart is totally needed, but I did it anyway).

Now when you are on a new terminal window and type “python” it should say version 2.6.5. You will have to reinstall any python bits and pieces that you have installed previously – but make sure that you reinstall PIP and easy_install – they will use the old version of Python if you are not careful.

Now go into the directory of psycopg2-2.2.1 that you downloaded earlier.

Change setup.cfg so it looks like the following:

[cc lang="bash"]
[build_ext]
define=PSYCOPG_EXTENSIONS,PSYCOPG_NEW_BOOLEAN,HAVE_PQFREEMEM,HAVE_PQPROTOCOL3

# PSYCOPG_EXTENSIONS enables extensions to PEP-249 (you really want this)
# PSYCOPG_DISPLAY_SIZE enable display size calculation (a little slower)
# HAVE_PQFREEMEM should be defined on PostgreSQL >= 7.4
# HAVE_PQPROTOCOL3 should be defined on PostgreSQL >= 7.4
# PSYCOPG_DEBUG can be added to enable verbose debug information
# PSYCOPG_OWN_QUOTING can be added, but it is deprecated (will go away in 2.1)
# PSYCOPG_NEW_BOOLEAN to format booleans as true/false vs ‘t’/'f’

# Set to 1 to use Python datatime objects for default date/time representation.
use_pydatetime=1

# If the build system does not find the mx.DateTime headers, try
# uncommenting the following line and setting its value to the right path.
#mx_include_dir=

# For Windows only:
# Set to 1 if the PostgreSQL library was built with OpenSSL.
# Required to link in OpenSSL libraries and dependencies.
have_ssl=0

# Statically link against the postgresql client library.
static_libpq=0

# “pg_config” is the preferred method to locate PostgreSQL headers and
# libraries needed to build psycopg2. If pg_config is not in the path or
# is installed under a different name uncomment the following option and
# set it to the pg_config full path.
pg_config=/Library/PostgreSQL/8.4/bin/pg_config

# If “pg_config” is not available, “include_dirs” can be used to locate
# postgresql headers and libraries. Some extra checks on sys.platform will
# still be done in setup.py.
# The next line is the default as used on psycopg author Debian laptop:
#include_dirs=/usr/local/lib

# Uncomment next line on Mandrake 10.x (and comment previous ones):
#include_dirs=/usr/include/pgsql/8.0:/usr/include/pgsql/8.0/server

# Uncomment next line on SUSE 9.3 (and comment previous ones):
#include_dirs=/usr/include/pgsql:/usr/include/pgsql/server

# If postgresql is installed somewhere weird (i.e., not in your runtime library
# path like /usr/lib), just add the right path in “library_dirs” and any extra
# libraries required to link in “libraries”.
library_dirs=/Library/PostgreSQL/8.4/lib
libraries=/usr/lib
[/cc]

Now run the installation using the new python:

[cc lang="bash"]python setup.py install[/cc]

This should install. You should be able to do “import psycopg2″ from the python prompt. Now install all the libraries and python modules that you need, and create the Postgresql database that you want (pgadmin3 is the best tool for this).

Please note this was done on a OS X Snow Leopard 10.6.4 with XCode and the 10.4 libraries installed.