The Django and Ubuntu Intrepid Almanac

February 13, 2009. Filed under django 72 apache 5 nginx 5 ubuntu 4 postgresql 4 memcache 3 mod_wsgi 2

It's been more than a year since my first endeavor to setup a healthy environment to deploy Django (as I was rudely reminded by the Ubuntu repositories for my server's version ceasing to function). In that time I've learned a lot, and have also stretched my original setup to its breaking point, but inevitably the day comes for a fresh deployment with better configurations and more flexible folder layouts.

From my last server setup guide, a number have things have remained the same: still using Ubuntu, Memcached, Postgres and still using Nginx as a proxy server infront of Apache2. A number of things have changed as well: using mod_wsgi over mod_python, cmemcache over python-memcached, and a more intentional folder layout along with virtualenv to make it straightforward to host multiple projects and domains (including some serving only static files or PHP scripts).

More than just an update, I've also included a few side-quests like using your server as a remote Git repository over SSH, and installing pluggable Django libraries. Finally, in the vein of my previous tutorial, I tried to include every keystroke required to transform a naked Ubuntu Intrepid server into a full-featured multi-site Django-loving server.

Please let me know if you run into any problems, or have suggestions on improvements!

Setting up your Server

  1. Somehow get a Ubuntu Intrepid server or VPS. (Perhaps go to your SliceHost console and request a new Slice running Ubuntu Intrepid (8.10). ;)

  2. Write down root password and IP address for your box.

  3. SSH into your server.

    ssh root@
  4. Update your apt-get sources.

    apt-get upgrade
  5. Make sure there is an editor that suits your taste available on the system. Vim is pre-installed, but I prefer Emacs...

    apt-get install emacs
  6. Setup any non-root accounts you want, and one for Django.

    useradd django
    mkdir /home/django
    chown django:django /home/django
    useradd will
    mkdir /home/will
    chown will:will /home/will
    passwd will
  7. Unless you like sh, change your default shell to something more humane.

    chsh root -s /bin/bash
    chsh will -s /bin/bash
    chsh django -s /bin/bash

    (These changes won't be applied until you log in the next time.)

  8. Give your account (but not Django) root permissions.


    Then use the down arrow for find a line that looks like this:

    root     ALL=(ALL) ALL

    Replicate that with your username:

    root     ALL=(ALL) ALL
    will     ALL=(ALL) ALL
  9. Open a second terminal (leave the first one logged in while we keep configuring SSH, incase something goes horribly awry), and SSH in as your non-root user.

    ssh will@

    Verify it works, then exit back to your system.

  10. Now it's time to setup password-less login. Return to your home system,

    scp ~/.ssh/ will@
    ssh will@
    mkdir .ssh
    mv .ssh/authorized_keys
    chmod go-w ~/.ssh/authorized_keys ~/.ssh/

    Now exit and ssh back in. You should have been logged in without needing to supply your password.

  11. Now it's time to restrict ssh a bit.

    sudo groupadd sshers
    sudo usermod -a -Gsshers will
    sudo emacs /etc/ssh/shhd_config

    Make these changes:

    #X11Forwarding yes
    X11Forwarding no

    And add these lines to the end of the file:

    UseDNS no
    AllowGroups sshers

    Save the file, and then restart the ssh service.

     sudo /etc/init.d/ssh restart

    exit your VPS, and once again try logging back in. If that worked, again open /etc/ssh/sshd_config.

     sudo emacs /etc/ssh/sshd_config

    And append this line to the end of the file:

    PasswordAuthentication no

    And once again restart the ssh service.

    sudo /etc/init.d/ssh restart

    Finally, disable password access to the root account.

    sudo passwd -l root

    Now your VPS is only accessible through your approved account from machines with the correct SSH key.

  12. Now it's time to start installing some general libraries.

    sudo apt-get install subversion git-core gcc curl
    sudo apt-get install build-essential python-dev python-setuptools
    sudo apt-get install python-egenix-mxdatetime memcached postfix
  13. Now it's time to configure Postgres, first we need to grab some libraries.

    sudo apt-get install postgresql-8.3 postgresql-server-dev-8.3
    sudo apt-get install postgresql-8.3 postgresql-server-dev-8.3

    Rather than a typo, I really did need to run the above command twice for a successful installation.

    Next we need to configure the postgres user.

    sudo -u postgres psql template1
    ALTER USER postgres WITH PASSWORD 'password';

    (Make sure that you used a real password, rather than 'password' in the above example.)

    We also need to modify Postgres' configuration file.

    sudo emacs /etc/postgresql/8.3/main/pg_hba.conf

    Move to the bottom of the file, and comment out (add a # at the beginning of the line) all lines which begin with host. (This prevents external access to your database.)

    It should look like this:

    # Database administrative login by UNIX sockets
    local   all         postgres                      ident sameuser
    # "local" is for Unix domain socket connections only
    local   all         all                            password
    # IPv4 local connections:
    #host    all         all          md5    
    # IPv6 local connections:
    #host    all         all         ::1/128               md5

    Note that we switched ident sameuser to password for the second local line! Then restart Postgres to have it reload its settings.

    sudo /etc/init.d/postgresql-8.3 restart

    And there we have it, Postgres is setup and functioning.

  14. And now it's time to setup memcached. This is a two part process. First we need to start memcached, which is very easy, and second we need to build cmemcache, which is a bit harder.

    sudo memcached -u www-data -p 11211 -m 32 -d

    That runs memcached on the standard port, with the standard www-data user (standard for Ubuntu, that is), with 32 megabytes of ram. You might want to allocate more, depending on how much data you're expecting to store in memcached, and the size of your VPS.

    Next we need to setup cmemcache. First grab libmemcache,

    sudo apt-get install libmemcache-dev

    Next we actually build cmemcache.

    su django
    mkdir ~/libs/
    cd ~/libs/
    tar -xjvf cmemcache-0.95.tar.bz2
    cd cmemcache-0.95

    We'll need to actually install the library later when we setup our virtualenv.

    If you are unable to install cmemcache, follow these instructions.

    Alternatively, if you want to use the Python based python-memcached (runs a bit slower, but no C module to build and install), you can do that as follows.

    tar -zxvf python-memcached-latest.tar
    cd python-memcached-1.43
    # later on use python install inside virtualenv

    The preference for cmemcache is strictly based on speed. But python-memcached is still quite usable, and if the choice is between python-memcached and not running memcached as your caching backend, picking python-memcached is the clear winner.

  15. Next it's time to setup Nginx, which is a lightweight server we'll use to serve static content as well as proxy requests to Apache21.

    sudo apt-get install nginx

    We can quickly verify that it installed correctly.

    sudo /etc/init.d/nginx start

    Should return:

    <title>Welcome to nginx!</title>
    <body bgcolor="white" text="black">
    <center><h1>Welcome to nginx!</h1></center>

    Now we're going to modify the nginx.conf file a bit (based on the advice here).

    sudo emacs /etc/nginx/nginx.conf

    Right now we only need to make two minor changes, first change line 2 to use four worker processes,

    worker_processes 4;

    And next uncomment line 18,

    tcp_nopush on;

    We won't need to edit this file again, because of the last line,

    include /etc/nginx/sites-enabled/*;

    which makes it easy to declare site specific settings in their own files, and really cuts down on clutter in nginx.conf.

    We also need to create a proxy.conf file, which is a common Nginx practice for keeping nginx.conf clean.

    sudo emacs /etc/nginx/proxy.conf

    Which should contain these lines:

    # proxy.conf
    proxy_redirect          off;
    proxy_set_header        Host            $host;
    proxy_set_header        X-Real-IP       $remote_addr;
    proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
    client_max_body_size    10m;
    client_body_buffer_size 128k;
    proxy_connect_timeout   90;
    proxy_send_timeout      90;
    proxy_read_timeout      90;
    proxy_buffers           32 4k;

    Finally, stop and start Nginx (I don't know if this is still the case with the current version of Nginx, but historically I've never gotten it to reload a configuration file with just a reload or restart).

    sudo /etc/init.d/nginx stop
    sudo /etc/init.d/nginx start
  16. Now it's time to setup Apache2. First we grab the necessary libraries.

    sudo apt-get install apache2 libapache2-mod-wsgi

    Because we already have Nginx bound on port 80 it will throw a bit of a complaint

    (98)Address already in use: make_sock: could not bind to address
    no listening sockets available, shutting down
    Unable to open logs!
     invoke-rc.d: initscript apache2, action "start" failed.

    But no worries, we're going to be alright.

    The first thing we need to do is to setup Apache to run on the internal instead of on port 80.

    sudo emacs /etc/apache2/ports.confg

    Then modify these lines:


    We also want to make a modification to apache2.conf:

    sudo emacs /etc/apache2/apache2.conf

    Search for KeepAlive (it's around line 77), and change its value as follows:

    # KeepAlive On
    KeepAlive Off

    We do this because Nginx doesn't yet support the KeepAlive option. Now let's try starting Apache, again.

    sudo apache2ctl start

    Which may complain yet, again.

    apache2: Could not reliably determine the server's fully qualified domain name, using for ServerName
    [Thu Feb 12 17:14:46 2009] [warn] NameVirtualHost has no VirtualHosts

    But once, again, no worries, it's just a phase in the process.

  17. Finally we get to actually grab Django. Switch over to the /home/django folder we created earlier, and it's time for us to create some folders.

    su django
    mkdir domains
    mkdir libs
    mkdir .python-eggs

    Then let's do some permissions wrangling.

    sudo chown django:www-data .python-eggs
    sudo chmod g+w .python-eggs/
    sudo usermod -a -G www-data django
    sudo usermod -a -G www-data will
    sudo chgrp -R www-data /home/django/domains
    sudo chmod -R 2750 /home/django/domains

    Then let's grab the Django source code.

     cd libs
    svn co django

    Normally this would be a great time to symlink the packages into /usr/lib/python2.5/site-packages, but we're going to go a slightly different route instead and use virtualenv to encapsulate each of our projects.

    sudo easy_install virtualenv

    We'll take care of setting up virtualenv later, because it is done on a per-project basis.

  18. Our last step of generic preparation is to remove the default sites for both Apache and Nginx. With properly configured setup files for Nginx/Apache (we'll get to those soon ;) you won't run into many situations where you are accidentally showing the default pages, but they often will be displayed when visiting the page directly via it's IP address.

    While that isn't a terribly common event, it's probably undesirable nonetheless.

    sudo rm /etc/apache2/sites-enabled/default
    sudo rm /etc/apache2/sites-enabled/default
    sudo rm /etc/apache2/sites-enabled/000-default
    sudo rm /etc/nginx/sites-enabled/default 

    You'll need to restart the server for the changes to take effect.

    sudo /etc/init.d/nginx stop
    sudo /etc/init.d/nginx start
    sudo apache2ctl graceful
  19. Now we're going to actually setup a Django project. You'll be able to follow this same process to setup any number of Django projects later.

    1. First we need to create the folders for each domain. (I'll walk through this example using the domain, which was hosting an old project long since forgotten.)

      cd /home/django/domains
      sudo mkdir -p{public,log}
      sudo mkdir
      sudo chown -R django:www-data

      Some people like to have private and backup directories as well. Feel free to add your own special spice.

      • The public directory will be for publicly accessible files served by Nginx.
      • The log directory will store the Apache and Nginx logs for the domain.
      • We'll keep the project dir in the folder itself. (And the mod_wsgi config file in the project directory.)
    2. Now we need to setup a virtual environment for the project. If you haven't experiment with virtualenv, it makes it easy to sandbox your projects, so that each project can have different versions of libraries (Django 0.97pre for one project, Django 1.1 for another, Pygments 0.6 for one project, Pygments 1.1 for another, etc).

      This makes it an essential tool for developing or hosting multiple applications on one server.

      Note that we previously downloaded Django from SVN into the folder /home/django/libs/django.

      su django
      cd ~/domains/
      virtualenv --no-site-packages --unzip-setuptools
      cd ~/libs/django
      ~/domains/ install
      cd ~/libs/cmemcache-0.95
      ~/domains/ install

      You will also need to install psycopg2 (Python interface for Postgres).

      cd domains/
      ./easy_install egenix-mx-base
      ./easy_install psycopg2

      For packages without easy_install or support, you can symlink them into the virtual environment's site-packages folder:

      cd ~/libs/
      ln -s `pwd`/some_app /home/django/domains/

      Now that we've done our installations, go ahead and test it out.


      Then try importing it.

      >>> import django

      Whereas if you do that with the normal python:


      You get a different experience:

      >>> import django
      Traceback (most recent call last):
        File "<stdin>", line 1, in <module>
      ImportError: No module named django

      Now that we're done setting up the virtualenv go ahead and exit the django user.

    3. Next we need to create an Nginx virtual host for the domain.

      sudo emacs /etc/nginx/sites-available/

      If you wish to strip www from your domain, you should begin the file with this server directive:

      You'll need to update to be the IP address of your VPS/server.

      server {
          rewrite ^/(.*)$1 permanent;

      Regardless, the remainder of the file should be in this format:

      You'll need to update to be the IP address of your VPS/server.

      server {
          access_log /home/django/domains/;
          error_log /home/django/domains/;
          location / {
              include       /etc/nginx/proxy.conf;
          location /media/ {
                  root   /home/django/domains/;
                  expires 1d;

      Note that if your static media changes frequently, then you'll want to remove the expires 1d setting. That adds a header saying that such content will not change for the next day, which means that the browser won't try to refetch it for that period of time.

      This means faster page loads for your users, and lower server load for you. If you want the benefits of future expires, but have frequently changing media, then the standard solution is to add a version number for your media files (me.1.png, me.2.png, etc). django-compress provides some support for automating the versioning process for JavaScript and CSS files (and is highly recommended).

      Next we need to let Nginx know that the site is enabled, by symlinking it from sites-available to sites-enabled.

      sudo ln -s /etc/nginx/sites-available/ /etc/nginx/sites-enabled/

      Finally you'll need to stop and start Nginx before Nginx knows that this virtual environment exists.

      sudo /etc/init.d/nginx stop
      sudo /etc/init.d/nginx start

      And you're done configuring Nginx.

    4. Now let's quickly create a default Django project.

      su django
      cd ~/domains/
      ./ startproject hello
    5. Link the Django admin media into the public/media/ folder.

      sudo ln -s /home/django/domains/ /home/django/domains/

      Then update the ADMIN_MEDIA_PREFIX setting in your hello/ file

      sudo emacs /home/django/domains/

      to look like this

      #ADMIN_MEDIA_PREFIX = '/media/'      
      ADMIN_MEDIA_PREFIX = '/media/admin/'
    6. Next let's create a Postgres database for this project to use.

      sudo su postgres
      createuser -P pg_substanceis
      # should not be a superuser
      # should not be able to create databases
      # should not be able to create more new roles
      createdb --encoding=UNICODE db_substanceis -O pg_substanceis

      Then we need to update the file with the proper settings.

      sudo emacs hello/

      And modify the DATABASE_* fields to look like this:

      DATABASE_ENGINE = 'postgresql_psycopg2'
      DATABASE_NAME = 'db_substanceis'
      DATABASE_USER = 'pg_substanceis'                   
      DATABASE_PASSWORD = '123456789'                   
      DATABASE_HOST = ''                   
      DATABASE_PORT = ''

      Finally let's try to sync the database to verify that we got the settings correct.

      ~/domains/ hello/ syncdb

      Assuming you entered your username, database and password correctly you should see something familiar.

      Creating table auth_permission
      Creating table auth_group
      Creating table auth_user
      Creating table auth_message
      Creating table django_content_type
      Creating table django_session
      Creating table django_site
    7. Our next step is to setup the project to memcached as its caching backend.

      sudo emacs hello/

      And add these settings:

      CACHE_BACKEND =	'memcached://'
      CACHE_MIDDLEWARE_KEY_PREFIX = 'sis' # SubstanceIS                  

      You'll have to decide what values of CACHE_MIDDLEWARE_SECONDS and CACHE_MIDDLEWARE_ANONYMOUS_ONLY are appropriate for the particular project you're configuring. There is more information available here.

      The value of CACHE_MIDDLEWARE_KEY_PREFIX should be unique for each project using memcached.

    8. After a tremendous journey, we reach the final step: configuring mod_wsgi and Apache to serve our project.

      sudo emacs /home/django/domains/

      And add this code to hello.wsgi:

      ALLDIRS = ['/home/django/domains/']
      # note that the above directory depends on the locale of your virtualenv,
      # and will thus be *different for each project!*
      import os
      import sys
      import site
      prev_sys_path = list(sys.path)
      for directory in ALLDIRS:
      new_sys_path = []
      for item in list(sys.path):
          if item not in prev_sys_path:
      sys.path[:0] = new_sys_path
      # this will also be different for each project!
      os.environ['PYTHON_EGG_CACHE'] = '/home/django/.python-eggs'
      os.environ['DJANGO_SETTINGS_MODULE'] = 'settings'
      import django.core.handlers.wsgi
      application = django.core.handlers.wsgi.WSGIHandler()

      Next we need to create a virtual host for our domain.

      sudo emacs /etc/apache2/sites-available/

      And add this VirtualHost definition.

          <Directory /home/django/domains/>
              Order deny,allow
              Allow from all
          LogLevel warn
          ErrorLog  /home/django/domains/
          CustomLog /home/django/domains/ combined
          WSGIDaemonProcess user=www-data group=www-data threads=25
          WSGIScriptAlias / /home/django/domains/

      And finally enable

      sudo ln -s /etc/apache2/sites-available/ /etc/apache2/sites-enabled/

      And restart Apache2 to have it pick up the new site.

      sudo apache2ctl graceful

    You can repeat these steps to host as many projects as you want.

Some Additional Options

At this point your mod_wsgi server is setup and fully functional, but here are a couple of suggestions for things you might want to do with your fresh server.

  1. At the moment if you go to your VPS's ip address, then you'll notice that it isn't serving anything at all. For most people that is probably the correct behavior, but you might want to serve a static page when they reach your VPS via its IP address (also, you can do the same thing to serve static content for some of your domains, rather than a full Django project).

    First we need to create some folders

    sudo mkdir /home/django/domains/default
    sudo mkdir -p /home/django/domains/default/{public,logs}

    Then create an index page

    sudo emacs /home/django/domains/default/public/index.html

    with this html

    <body><p>Hi. Welcome to my default site.</p></body>

    Now we need to configure the Nginx site

    sudo emacs /etc/nginx/sites-available/default

    with these settings (you'll need to change to your VPS's ip address):

    server {
            listen   80;
            access_log /home/django/domains/default/log/access.log;
            error_log /home/django/domains/default/log/error.log;
            location / {
                     root /home/django/domains/default/public;
                     index index.html;

    Then symlink it into the sites-enabled folder.

    sudo ln -s /etc/nginx/sites-available/default /etc/nginx/sites-enabled/default

    And finally we need to stop-start nginx.

    sudo /etc/init.d/nginx stop
    sudo /etc/init.d/nginx start

    Now if you navigate to your VPS's ip address in a browser, you'll see your static site. (Note that browsers typically limit the number of concurrent http connections to one domain, so you can speed up page load for heavy pages by using these steps to setup a media only, etc--for your Django sites and serving static media from them instead of the same domain as your site.)

  2. One of the most helpful packages around is PIL, which supports a wide variety of image manipulation techniques. It's also used by some pieces of Django. It seems to be improperly packaged for installation with easy_install, but it is still straightforward to install.

    su django
    cd ~/libs
    tar -xzvf Imaging-*
    cd Imaging-1.1.6
    /domains/ install

    And now the virtual environment will have access to the Image module:

    >>> import Image
  3. The more Django development you do, the more you'll be creating pluggable applications and reusing those made by others, so it's a good process to be familiar with. So let's take a few moments to install a couple common reusable Django applications developed by the Django community.

    One of the downsides of using virtualenv is that you'll need to install pluggable apps into each of your environments one at a time. Of course, it also means that you can freeze a production site to use a certian version of a pluggable app while another plunges ahead into the unknown.

    It is important to note the benefits of installing with or easy_install in conjunction with virtualenv. It means that you'll get a frozen copy of the code, and even if you update the source code later on, the environments won't be running different code.

    On the other hand, if you're actively developing code, then you would be better off symlinking the code into the environment's site-packages directory so that you don't have to redeploy the code on changes.

    Both approaches have their place, just be careful to use them appropriately.

    First let's install django-compress, which I mentioned earlier and helps automating the merging and compression of JavaScript and CSS files.

    su django
    cd /django/home/libs
    sudo svn checkout django-compress
    cd django-compress
    ~/domains/ install

    Now let's make sure it's usable.


    Then within Python try

    >>> import compress

    As long as you don't get an error, then it installed correctly.

    Next let's install django-faq, which is a pluggable application for creating faqs on websites (managable via the Django admin).

    Just for fun, let's symlink it in (more convenient for active development) instead of using

    cd /django/home/libs
    sudo git clone git://
    cd django-faq
    ln -s `pwd`/faq /home/django/domains/

    Finally let's install django-mailer, which supplements Django with a more robust mailing framework.

    cd /django/home/libs
    sudo svn checkout django-mailer
    cd django-mailer
    sudo ln -s `pwd`/mailer /home/django/domains/

    Wait, is it okay to recommend one of my projects? Ok. You should consider taking a look at django-monetize, which supports dynamically serving different kinds of advertisements (and requests for donations, etc) based on a page's context.

    cd /django/home/libs
    sudo git clone git://
    cd django-monetize
    sudo ln -s `pwd`/django_monetize /home/django/domains/

    Moving past these pluggable apps, there are also a number of projects which have expanded to be something akin to mini-frameworks on top of Django. They provide a great amount of functionality, but often at the expense of somewhat non-standard installation patterns.

    Two of the more popular are the social framework Pinax ( installation ), and the webshop platform Satchmo ( installation ).

  4. Another thing I like doing is to use my VPS as a Git repository, which is--thanks to the manifold joys of DVCS--unexpectedly simple.

    mkdir git
    mkdir git/scripts
    cd git/scripts
    echo "# a test" >
    git init --bare

    Then exit your VPS and return to your machine. From your home machine you can clone that repository by using SSH.

    mkdir my_repo
    cd my_repo
    git init
    echo "a test" >
    git add
    git commit -m "Initial commit."
    git remote add origin ssh://
    git push origin master

    Note that using the --bare keyword means that repository will keep all its files in its .git folder, so you won't be able directly interact with your files in that hosted repository. If you do want to interact with the files from that git repository on your machine, you should git clone from the origin repository and then push and pull like you normally would.

    (If you're wanting to use Git as a mechanism for pushing code for your Django projects, you might want to read up about using something like Fabric.)

  5. Although admittedly not as glamorous as being a Djangonaut, there are a number of situations where you may want Apache to also run some php scripts for you.

    sudo apt-get install php5
    sudo apache2ctl graceful

    Then just place .php files in a directory served by Apache (not by Nginx), and they'll be executed. You'll probably want a combination of the Django and static Nginx configuration files that we've put together through the course of this walkthrough.


These articles draw heavily from my previous nginx+apache+mod_python+django and lighttpd+apache+mod_python+django articles from last year, but have taken advantage of new resources as well.

  1. The mod_wsgi page on Google Code is a great launching point.
  2. As is the mod_wsgi page on integrating with Django.
  3. Setting up Nginx Virtual Hosts on Ubuntu.
  4. A full example of an Nginx config file.
  5. Setting up mod_wsgi with Django.
  6. Having Nginx serve ip addresses.
  7. Using virtualenv with mod_wsgi.

Many thanks to those who created these above resources that made this all possible.

Updated 2/13: Virtual Env

Tom Schreiber mentioned that virtualenv is a great tool for developing multiple projects with differing versions of libraries, and on his recommendation I updated the walkthrough to include it. Thanks Tom!

Updated, Again 2/13: Misc. Tweaks

A helpful Redditor pointed out some ways for improving. Particularly using chsh for changing shells (instead of manually editing the /etc/passwd file), and having Apache listen only to instead of binding to *:8080, which makes it externally accessible, and having Nginx listen to external-ip:80 instead of *:80.

  1. Hey kids, did you know that Nginx is pronounced "engine x". I sure as hell didn't. I actually had a coworker tell me how it was pronounced a while ago, but without seeing "engine x" written it never quite worked for me. Now I get it. Finally.