Tim Child

This is something that I didn’t think that I would be writing in 2012 – we are switching to a Java based applet for our uploader.

The problem has been that browsers and their operating systems handle uploads differently. Really differently. Evening using a project like plupload, you can’t polyfill over the cracks. There are new APIs coming out (if you want to call it HTML5, not strictly true though), but to handle backwards compatibility, including IE9, there needs to be a mechanism to make it generic and the only thing is Java.

• Flash can be ok, but there are limitations in filesize,
• Javascript depends on the browser implementation.
• Gears & Silverlight – not exactly something that I want to deploy.
• Java – the cross platform way.
• Depending on the browsers built in widgets – Not possible for large files.

We need to transfer large files of over 2Gb, and some browsers will actually try and load the entire file into memory before transferring. Certain death of the browser on old machines.

Java gives us chunking, it gives us a cross platform implementation, it gives us a easy testing mechanism. 2012 – roll out your applets.

Really interesting article about Javascript MVC frameworks at CodeBrief.

Personally and for work I am totally into backbone.js. It is changing the way that I work with Javascript in my web applications for the better by bringing greater structure and organisation to my code and having a common methodology in the way that I approach server-browser communications.

The only problem that I have is that having a MVC stack in the browser is fine, but then we have the same on the server side with Rails (ok not exactly MVC), Django, Node.js or what have you. We end up with MVCMVC or rather MCVCMV, and this is of course a pattern that causes code duplication if we are not careful.

I might look into creating backbone.js models from Django at some point, but I am not sure that this is the correct route. Obviously the way that GWT and PyJamas work becomes more compelling. Well for me GWT is not that compelling as I don’t really want to have to work on frontend in Java, I don’t think there is the control that one needs over the presentation and I can’t stand Eclipse/NetBeans/xxxx IDE’s as they are slow and cumbersome.

Regarding the Codebrief article ember.js came out on top, and I can see the negatives of backbone.js in its somewhat boilerplate approach though I think this can be minimised somewhat.

Personally I think my goals going forward for Javascript is:

1. Find a decent testing enviroment that I am happy with
2. Loading of modules should be optional, and I need to work to convert my existing application to load intelligently our backbone.js models and views
3. Greater reuse or automatic generation of code

I am intending to write about all of this on this blog in the near future as I come up with more conclusions.

There is a really interesting article over on Ilya Grigorik’s blog titled . Be sure to read the comments if you are interested in how AOL built a similar system many moons ago.

I have been thinking about next generation web stacks, with proper (M)VC in the browser, communication with server using SPDY / WebSockets / SSE or similar with SSL, and then the topology of the stack in the background being event driven and that article has some really nice points. Kinda interested in playing with 0MQ now and perhaps hooking it up to Go, Erlang or just plain old Gevent.

Probably wouldn’t use a NoSQL backend because there is little reason most of the time.

The hackernews article is here: http://news.ycombinator.com/item?id=3481140

Damn I feel old fashioned having this blog on Apache | PHP | MySQL. At least my work environment is Nginx | Gunicorn | Python | Postgresql

So another new year has started but I continue to improve and refine my workflow and tooling.

Firstly git. The more I learn with git, the more I like it, and the more I get into it. I like the fact that I don’t need any other tool, if I was using Mercurial I would still have to use Git because of the amount of excellent code being published into GitHub. For example “git whatchanged” or however.

ZSH – I think its maybe because I invested the effort to configure it, put together my own theme but I am getting lots of improvements out of this.

Dotfiles in GitHub. I have more than one machine nowdays, in fact I generally work on 6 machines (Laptop, Virtual Ubuntu on Laptop, General purpose SSH server, home server, home OS X box and a work VMWare image), so having generic dotfiles with my configuration, ZSH configuration, Vim config and more really helps.

Working with django, multiple branches, I have switched pretty much fully to virtualenv and virtualenvwrapper. I now have one command for switching, with scripts that can move my config files etc.

SublimeText2 is pretty much my editor of choice on GUI with Vim on the server.

iterm2 for terminal on OSX. TMux for remote servers, so if I am using something more dumb than iTerm2 I still have multiple panes. This together with trying to get more and more keyboard shortcuts (even in Apple Mail), and less mouse usage is pretty damn good.

At some point I am going to start blogging more about other bits and pieces here I think, and move a more personal blog somewhere else. Either more in Facebook or into GooglePlus. This will all be linked together, as I have many separate profiles all over the place now.

I recently had the opportunity to look at what I would install on a vanilla OS X system and its was quite interesting to look at what I depend on and if I didn’t have it, what would be on my Christmas software list. So here goes:

1. Git: Probably not the first thing that I would install, but it wouldn’t be the last. Via the proper GIT installer
2. Python 2.7: Wouldn’t bother with the OS X version, I would go straight for the version from python.org
3. Postgresql 9.x: Of course I need a database.
4. Brew: Homebrew is probably the best way to install lots of little scripts and things
5. XCode: For development or installing lots of interesting software, you are going to need it at some point.
6. MacVim: Fast becoming my favourite editor.
7. Quicksilver: Good launcher.
8. iTerm2: Terminal done correctly.
9. Google Chrome: Love safari, but this is great as well.
10. Scroll Reverser (Only if using Snow Leopard or previous).
11. Evernote: Great task management.
12. DropBox: Good way to sync multiple documents.

I think the weirdest thing about this list is that there is no Office, no mail client (I can use Mail or Google Mail), none of the productivity Apps that I used to use. Increasingly I am turning to try and run everything within Terminal or iTerm (MacVim just makes me comfortable, but I spend large amounts of time in plain old vim). The other odd or perhaps not is that everything is available at no cost.

Having started to change my environment quite a bit by changing from bash to zsh, Textmate (ok I am still using it quite a bit) to Vim and MacVim with Janus, I thought that I would join what seems to be a popular movement and share my dot files on GitHub.

https://github.com/timc3/dotfiles

Never thought that I would be saying this, but I am really liking Javascript at the moment.

It seems that no matter how many pixels I have to play with I could always use more. Consider this number of apps that I have open a day:

Browser window with debugging pane (1200 x 1200)
skype (220×1000 high) with conversation window (400×600)
Mail (1000×800)
Text mate or vim (typically 1600×1200)
Terminal windows – at least two. One for Git and one for python shell. (1600×1200)
2nd browser window for looking stuff up though I can use tabs for them.

Could also do with having an irc client, finder and adium but I usually close them.

It all equals too many pixels

The installer for Postgresql on Lion is supposedly “unsupported”, which means that they haven’t got it working yet. Looking at it quickly is seems to be the way that the postgres user is being created using dscl.

To get it to work, manually create a postgres user from under System Preferences:

Then run the installer as per normal. I used the Postgres 9 installer.

I also upgraded python to 2.7.2 and installed psycopg2 pointing to pg_config under /Library/PostgreSQL/9.0/bin/pg_config.

Looking back over the last posts I realize that I haven’t been talking much about the software and and code that I have been using every day and some of it is really quite interesting.

Python. I use python a lot, it’s my preferred go to language to solve a lot of problems. It has great libraries, an awesome community and the language itself is compact and non verbose. We are using Django heavily in our solutions and its really enjoyable to work with, and I never feel that coding is a chore. Coupled with the fact that we can target Java servers with Jython, .Net with IronPython and have an amazing solution in PyPy for python running on Python I can see the love affair continue.

Eventlet, Gevent and Meinheld. Python networking libraries (actually Meinheld is a WSGI server) that work in an event’d manner, all are really easy to work with, all are really fast. With these libraries its really easy to build highly performant services that scale well on standard architecture. For instance to write a websocket / comet server, I wouldn’t hesitate to use Eventlet or Gevent. Tornado looks like another nice Python alternative but I have never been happy with its lib curl dependancy.

Celery. Asynchronous task queue/job queue written in Python for python that can use message servers such as RabbitMQ. Really easy to build strong reliable services spanning multiple machines.

Hudson. Continuous integration server that is just seamless, I use it for building packages (python and Debian), running test frameworks and automatically building documentation which gets uploaded to Dropbox. Can’t recommend it strongly enough.

Linux KVM and Proxmox. Virtualisation has obviously been a part of most deployment and development cycles for some time now , but innovation is still being made. Having used VMWare, VMWare ESX, VirtualBox, Microsoft’s Hyper-V and others I have found what I believe to be the sweet point in Linux KVM – simple extensions to the Linux kernel which allows for running virtual machines on top of a normal Linux platform. I have moved my personal servers, and our work servers to KVM as much as possible and an easy integrated way of doing this is to run ProxMox.

Memcached. Memory caching server that I use with my goto web services stack – Nginx – Supervisord – Meinheld – Django – Memcached – Postgres. Simply just works.

Selenium. Automated browser testing, this is a product that no matter how much I use it, I feel that I can use it more than I do. Have been using the RC version to do browser testing against firefox and IE on Windows.

PostgreSQL. NoSQL is all very well and good, but for most serious projects I tend to turn to Postgres for its fully featured SQL compliance, its rock solid hosting of data and its well proven ability to scale.

RabbitMQ. RabbitMQ is a message queue that I tend to deploy with Python Celery to help perform offline processing of tasks.

Supervisord. Currently used for daemonising a setup.

Git. Distributed version control. I have tried most of the others, and before that been a long time user of SVN but Git installs on everything, has awesome support in GitHub, and I am really enjoying the extending merging and branching.

Redmine. For task tracking and bug fixing, this is the best tool that I have used. Migrated everything from trac to it with no problems, and have also migrated from Redmine backed by MySQL to PostgreSQL with no problems.

Glassfish. My goto Java Application Server. I hope to have more exposure to JBoss in the coming few months, but Glassfish has been good so far.

nginx. Can’t say enough good things about Nginx. Its fast, easy to configure, flexible, lightweight and hosts all our software at Cantemo with little problem.

HAProxy. When I need to load balance nginx or other services (including straight TCP/IP which nginx can’t load balance) I turn to HAProxy.

backbone.js. Almost all the Javascript that I am now creating is built upon underscore.js and backbone.js. There are many tutorials but this deserves another space all of its own.

In fact there are many things that I have left out that I am using everyday but these are some of the technologies, software and libraries that have been making my life easier and the systems that I have built more interesting for the past 18 months or so. I really ought to do a write up on each of them.