Rev Matt blog reviews projects about

Time for you to upgrade

Moving to django 1.8 and postgres from 1.6.4 and mysql on web faction

First and foremost I know it looks like crap on mobile, I was shooting for an iterative upgrade, responsive css will be later this week.

Environment

Environment setup was pretty simple, I do everything in virtual environments so I can use the python version I want and install site-packages. I set up a VM on my computer with CentOS 6 to mimic the production environment as best I could to make deployments smooth.

  • Set up a WSGI/Python3 app in the web faction control panel
  • cd webapps/rev
  • virtualenv .
  • source bin/activate
  • pip install django
  • django-admin startproject rev
  • vi apache2/conf/httpd.conf
    this is the only part that involves much effort, and it's just adding the paths to the webapp and site-packages to WSGIPythonPath and a script alias to the wsgi.py file.
  • deploy my files to the rev project folder, static files to a static app in the webfaction admin that maps to [url]/static
  • . apache2/bin/restart

Modifying django

Most of my custom code is fairly straightforward, all variations on a blog format really. Blog, Projects, Reviews, and the much simpler Quotes, I didn’t have to change anything for the code to work as I'd moved to class-based views during my last upgrade. I will probably do some refactoring to take advantage of the updated Model ._meta API and maybe play with Jinja2 now that the heavy lifting of the upgrade is done. I will probably also move the design CV to django-photologue rather than the very ugly flatpage it currently is.

Data conversion

The worst part of any migration is data conversion. I’m no database surgeon so it probably could have been done cleaner. Based on some info from EndPoint I managed to clean up the data types, but (and this might be specific to the version of mysqldump I have running) I had to go in and change all the UNIQUE declarations as well. mysqldump creates tables like “UNIQUE "content_type_id" ("content_type_id","codename”),” which postgres doesn’t like. Drop the chunk outside the parens and all is happy. Having done all this and started working on the actual nitty gritty of creating all the structures I realized it was largely unnecessary. I could syncdb with my updated app and it would create everything I needed and then it would just be a matter of minor tweaks to data to load everything I needed. This is why I often delay starting a project while I think through the steps in the background for a few days.
  • double the slashes on escape characters
  • :g/\\n/s//\\\\n/g
  • :g/\\r/s//\\\\r/g
  • prepend the escape indicator on string literals
  • :g/\,\’/s//,E\’/g
  • If you were using ints as booleans (no idea why I did that) you’ll need to wrap those in '
    In my specific case the data had author id then boolean, so I could search for [id],[int] and do replaces that way. Your experience will vary.

I did get to the point of scripting the drop database/create database/grant permissions tango

Note: the amount of ruby fiddling necessary to get pgloader to work was absolutely not worth it, I gave up 5 steps into getting pgloader to just plain run on OS X let alone actually connect to any database. If it involves ruby or perl, I’d rather write my own tools or do it by hand.

I also went for a much cleaner and simpler design, which took longer than the previous design just because it’s hard to make simple look good.

Overall it took a few hours each weekend over two weekends to get this done, it could've all been done in a single day if I'd focused solely on the upgrade project.

blog comments powered by Disqus

I read about an Eskimo hunter who asked the local missionary priest, ‘If I did not know about God and sin, would I go to hell?’ ‘No,’ said the priest, ‘not if you did not know.’ ‘Then why,’ asked the Eskimo earnestly, ‘did you tell me?’
- Annie Dillard

Archives

2014 ~ 2013 ~ 2012 ~ 2011 ~ 2010 ~ 2009 ~ 2008 ~ 2007 ~ 2006 ~ 2005 ~ 2004 ~ 2003 ~ 2002 ~ 2001 ~ 2000 ~ 1997