The Django and Ubuntu Intrepid Almanac

by Will Larson February 13, 2009 django ubuntu memcache postgresql apache nginx mod_wsgi

It's been more than a year since my first endeavor to setup a healthy environment to deploy Django (as I was rudely reminded by the Ubuntu repositories for my server's version ceasing to function). In that time I've learned a lot, and have also stretched my original setup to its breaking point, but inevitably the day comes for a fresh deployment with better configurations and more flexible folder layouts.

From my last server setup guide, a number have things have remained the same: still using Ubuntu, Memcached, Postgres and still using Nginx as a proxy server infront of Apache2. A number of things have changed as well: using mod_wsgi over mod_python, cmemcache over python-memcached, and a more intentional folder layout along with virtualenv to make it straightforward to host multiple projects and domains (including some serving only static files or PHP scripts).

More than just an update, I've also included a few side-quests like using your server as a remote Git repository over SSH, and installing pluggable Django libraries. Finally, in the vein of my previous tutorial, I tried to include every keystroke required to transform a naked Ubuntu Intrepid server into a full-featured multi-site Django-loving server.

Please let me know if you run into any problems, or have suggestions on improvements!

Setting up your Server

Somehow get a Ubuntu Intrepid server or VPS. (Perhaps go to your SliceHost console and request a new Slice running Ubuntu Intrepid (8.10). ;)
Write down root password and IP address for your box.
SSH into your server.
```
ssh root@255.255.255.255
```
Update your apt-get sources.
```
apt-get upgrade
```
Make sure there is an editor that suits your taste available on the system. Vim is pre-installed, but I prefer Emacs...
```
apt-get install emacs
```

Setup any non-root accounts you want, and one for Django.

useradd django
mkdir /home/django
chown django:django /home/django
useradd will
mkdir /home/will
chown will:will /home/will
passwd will

Unless you like sh, change your default shell to something more humane.
```
chsh root -s /bin/bash
chsh will -s /bin/bash
chsh django -s /bin/bash
```
(These changes won't be applied until you log in the next time.)
Give your account (but not Django) root permissions.
```
visudo
```
Then use the down arrow for find a line that looks like this:
```
root     ALL=(ALL) ALL
```
Replicate that with your username:
```
root     ALL=(ALL) ALL
will     ALL=(ALL) ALL
```
Open a second terminal (leave the first one logged in while we keep configuring SSH, incase something goes horribly awry), and SSH in as your non-root user.
```
ssh will@255.255.255.255
```
Verify it works, then exit back to your system.
Now it's time to setup password-less login. Return to your home system,
```
scp ~/.ssh/id_dsa.pub will@255.255.255.255:~/
ssh will@255.255.255.255
mkdir .ssh
mv id_dsa.pub .ssh/authorized_keys
chmod go-w ~/.ssh/authorized_keys ~/.ssh/
```
Now exit and ssh back in. You should have been logged in without needing to supply your password.
Now it's time to restrict ssh a bit.
```
sudo groupadd sshers
sudo usermod -a -Gsshers will
sudo emacs /etc/ssh/shhd_config
```
Make these changes:
```
#X11Forwarding yes
X11Forwarding no
```
And add these lines to the end of the file:
```
UseDNS no
AllowGroups sshers
```
Save the file, and then restart the ssh service.
```
 sudo /etc/init.d/ssh restart
```
exit your VPS, and once again try logging back in. If that worked, again open /etc/ssh/sshd_config.
```
 sudo emacs /etc/ssh/sshd_config
```
And append this line to the end of the file:
```
PasswordAuthentication no
```
And once again restart the ssh service.
```
sudo /etc/init.d/ssh restart
```
Finally, disable password access to the root account.
```
sudo passwd -l root
```
Now your VPS is only accessible through your approved account from machines with the correct SSH key.

Now it's time to start installing some general libraries.

sudo apt-get install subversion git-core gcc curl
sudo apt-get install build-essential python-dev python-setuptools
sudo apt-get install python-egenix-mxdatetime memcached postfix

Now it's time to configure Postgres, first we need to grab some libraries.

sudo apt-get install postgresql-8.3 postgresql-server-dev-8.3
sudo apt-get install postgresql-8.3 postgresql-server-dev-8.3

Rather than a typo, I really did need to run the above command twice for a successful installation.

Next we need to configure the postgres user.

sudo -u postgres psql template1
ALTER USER postgres WITH PASSWORD 'password';
 \q

(Make sure that you used a real password, rather than 'password' in the above example.)

We also need to modify Postgres' configuration file.

sudo emacs /etc/postgresql/8.3/main/pg_hba.conf

Move to the bottom of the file, and comment out (add a # at the beginning of the line) all lines which begin with host. (This prevents external access to your database.)

It should look like this:

# Database administrative login by UNIX sockets
local   all         postgres                      ident sameuser
# TYPE  DATABASE    USER        CIDR-ADDRESS          METHOD 
# "local" is for Unix domain socket connections only
local   all         all                            password
# IPv4 local connections:
#host    all         all         127.0.0.1/32          md5    
# IPv6 local connections:
#host    all         all         ::1/128               md5

Note that we switched ident sameuser to password for the second local line! Then restart Postgres to have it reload its settings.

sudo /etc/init.d/postgresql-8.3 restart

And there we have it, Postgres is setup and functioning.

And now it's time to setup memcached. This is a two part process. First we need to start memcached, which is very easy, and second we need to build cmemcache, which is a bit harder.
```
sudo memcached -u www-data -p 11211 -m 32 -d
```
That runs memcached on the standard port, with the standard www-data user (standard for Ubuntu, that is), with 32 megabytes of ram. You might want to allocate more, depending on how much data you're expecting to store in memcached, and the size of your VPS.

Next we need to setup cmemcache. First grab libmemcache,
```
sudo apt-get install libmemcache-dev
```
Next we actually build cmemcache.
```
su django
mkdir ~/libs/
cd ~/libs/
wget http://gijsbert.org/downloads/cmemcache/cmemcache-0.95.tar.bz2
tar -xjvf cmemcache-0.95.tar.bz2
cd cmemcache-0.95
exit
```
We'll need to actually install the library later when we setup our virtualenv.

If you are unable to install cmemcache, follow these instructions.

Alternatively, if you want to use the Python based python-memcached (runs a bit slower, but no C module to build and install), you can do that as follows.
```
wget ftp://ftp.tummy.com/pub/python-memcached/python-memcached-latest.tar.gz
tar -zxvf python-memcached-latest.tar
cd python-memcached-1.43
# later on use python setup.py install inside virtualenv
```
The preference for cmemcache is strictly based on speed. But python-memcached is still quite usable, and if the choice is between python-memcached and not running memcached as your caching backend, picking python-memcached is the clear winner.
Next it's time to setup Nginx, which is a lightweight server we'll use to serve static content as well as proxy requests to Apache2¹.
```
sudo apt-get install nginx
```
We can quickly verify that it installed correctly.
```
sudo /etc/init.d/nginx start
curl 127.0.0.1
```
Should return:
```
<html>
<head>
<title>Welcome to nginx!</title>
</head>
<body bgcolor="white" text="black">
<center><h1>Welcome to nginx!</h1></center>
</body>
</html>
```
Now we're going to modify the nginx.conf file a bit (based on the advice here).
```
sudo emacs /etc/nginx/nginx.conf
```
Right now we only need to make two minor changes, first change line 2 to use four worker processes,
```
worker_processes 4;
```
And next uncomment line 18,
```
tcp_nopush on;
```
We won't need to edit this file again, because of the last line,
```
include /etc/nginx/sites-enabled/*;
```
which makes it easy to declare site specific settings in their own files, and really cuts down on clutter in nginx.conf.

We also need to create a proxy.conf file, which is a common Nginx practice for keeping nginx.conf clean.
```
sudo emacs /etc/nginx/proxy.conf
```
Which should contain these lines:
```
# proxy.conf
proxy_redirect          off;
proxy_set_header        Host            $host;
proxy_set_header        X-Real-IP       $remote_addr;
proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
client_max_body_size    10m;
client_body_buffer_size 128k;
proxy_connect_timeout   90;
proxy_send_timeout      90;
proxy_read_timeout      90;
proxy_buffers           32 4k;
```
Finally, stop and start Nginx (I don't know if this is still the case with the current version of Nginx, but historically I've never gotten it to reload a configuration file with just a reload or restart).
```
sudo /etc/init.d/nginx stop
sudo /etc/init.d/nginx start
```
Now it's time to setup Apache2. First we grab the necessary libraries.
```
sudo apt-get install apache2 libapache2-mod-wsgi
```
Because we already have Nginx bound on port 80 it will throw a bit of a complaint
```
(98)Address already in use: make_sock: could not bind to address 0.0.0.0:80
no listening sockets available, shutting down
Unable to open logs
   ...fail!
 invoke-rc.d: initscript apache2, action "start" failed.
```
But no worries, we're going to be alright.

The first thing we need to do is to setup Apache to run on the internal 127.0.0.1:80 instead of on port 80.
```
sudo emacs /etc/apache2/ports.confg
```
Then modify these lines:
```
NameVirtualHost 127.0.0.1:80
Listen 127.0.0.1:80
```
We also want to make a modification to apache2.conf:
```
sudo emacs /etc/apache2/apache2.conf
```
Search for KeepAlive (it's around line 77), and change its value as follows:
```
# KeepAlive On
KeepAlive Off
```
We do this because Nginx doesn't yet support the KeepAlive option. Now let's try starting Apache, again.
```
sudo apache2ctl start
```
Which may complain yet, again.
```
apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1 for ServerName
[Thu Feb 12 17:14:46 2009] [warn] NameVirtualHost 127.0.0.1:80 has no VirtualHosts
```
But once, again, no worries, it's just a phase in the process.
Finally we get to actually grab Django. Switch over to the /home/django folder we created earlier, and it's time for us to create some folders.
```
su django
mkdir domains
mkdir libs
mkdir .python-eggs
exit
```
Then let's do some permissions wrangling.
```
sudo chown django:www-data .python-eggs
sudo chmod g+w .python-eggs/
sudo usermod -a -G www-data django
sudo usermod -a -G www-data will
sudo chgrp -R www-data /home/django/domains
sudo chmod -R 2750 /home/django/domains
```
Then let's grab the Django source code.
```
 cd libs
svn co http://code.djangoproject.com/svn/django/trunk/ django
```
Normally this would be a great time to symlink the packages into /usr/lib/python2.5/site-packages, but we're going to go a slightly different route instead and use virtualenv to encapsulate each of our projects.
```
sudo easy_install virtualenv
```
We'll take care of setting up virtualenv later, because it is done on a per-project basis.
Our last step of generic preparation is to remove the default sites for both Apache and Nginx. With properly configured setup files for Nginx/Apache (we'll get to those soon ;) you won't run into many situations where you are accidentally showing the default pages, but they often will be displayed when visiting the page directly via it's IP address.

While that isn't a terribly common event, it's probably undesirable nonetheless.
```
sudo rm /etc/apache2/sites-enabled/default
sudo rm /etc/apache2/sites-enabled/default
sudo rm /etc/apache2/sites-enabled/000-default
sudo rm /etc/nginx/sites-enabled/default 
```
You'll need to restart the server for the changes to take effect.
```
sudo /etc/init.d/nginx stop
sudo /etc/init.d/nginx start
sudo apache2ctl graceful
```

Now we're going to actually setup a Django project. You'll be able to follow this same process to setup any number of Django projects later.

First we need to create the folders for each domain. (I'll walk through this example using the domain substanceis.com, which was hosting an old project long since forgotten.)
```
cd /home/django/domains
mkdir substanceis.com
sudo mkdir -p substanceis.com/{public,log}
sudo mkdir substanceis.com/public/media
sudo chown -R django:www-data substanceis.com
```
Some people like to have private and backup directories as well. Feel free to add your own special spice.
- The public directory will be for publicly accessible files served by Nginx.
- The log directory will store the Apache and Nginx logs for the domain.
- We'll keep the project dir in the substanceis.com folder itself. (And the mod_wsgi config file in the project directory.)
Now we need to setup a virtual environment for the project. If you haven't experiment with virtualenv, it makes it easy to sandbox your projects, so that each project can have different versions of libraries (Django 0.97pre for one project, Django 1.1 for another, Pygments 0.6 for one project, Pygments 1.1 for another, etc).

This makes it an essential tool for developing or hosting multiple applications on one server.

Note that we previously downloaded Django from SVN into the folder /home/django/libs/django.
```
su django
cd ~/domains/substanceis.com
virtualenv --no-site-packages --unzip-setuptools substanceis.com
cd ~/libs/django
~/domains/substanceis.com/substanceis.com/bin/python setup.py install
cd ~/libs/cmemcache-0.95
~/domains/substanceis.com/substanceis.com/bin/python setup.py install
```
You will also need to install psycopg2 (Python interface for Postgres).
```
cd domains/substanceis.com/substanceis.com/bin/
./easy_install egenix-mx-base
./easy_install psycopg2
```
For packages without easy_install or setup.py support, you can symlink them into the virtual environment's site-packages folder:
```
cd ~/libs/
ln -s `pwd`/some_app /home/django/domains/substanceis.com/substanceis.com/lib/python2.5/site-packages/
```
Now that we've done our installations, go ahead and test it out.
```
~/domains/substanceis.com/substanceis.com/bin/python
```
Then try importing it.
```
>>> import django
>>> 
```
Whereas if you do that with the normal python:
```
cd
python
```
You get a different experience:
```
>>> import django
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named django
>>> 
```
Now that we're done setting up the virtualenv go ahead and exit the django user.
```
exit
```
Next we need to create an Nginx virtual host for the domain.
```
sudo emacs /etc/nginx/sites-available/substanceis.com
```
If you wish to strip www from your domain, you should begin the file with this server directive:

You'll need to update 255.255.255.255 to be the IP address of your VPS/server.
```
server {
    listen   255.255.255.255:80;
    server_name  www.substanceis.com;
    rewrite ^/(.*) http://substanceis.com/$1 permanent;
}
```
Regardless, the remainder of the file should be in this format:

You'll need to update 255.255.255.255 to be the IP address of your VPS/server.
```
server {
    listen   255.255.255.255:80;
    server_name www.substanceis.com substanceis.com;

    access_log /home/django/domains/substanceis.com/log/access.log;
    error_log /home/django/domains/substanceis.com/log/error.log;

    location / {
        proxy_pass    http://127.0.0.1:80/;
        include       /etc/nginx/proxy.conf;
    }

    location /media/ {
            root   /home/django/domains/substanceis.com/public/;
            expires 1d;
    }
}
```
Note that if your static media changes frequently, then you'll want to remove the expires 1d setting. That adds a header saying that such content will not change for the next day, which means that the browser won't try to refetch it for that period of time.

This means faster page loads for your users, and lower server load for you. If you want the benefits of future expires, but have frequently changing media, then the standard solution is to add a version number for your media files (me.1.png, me.2.png, etc). django-compress provides some support for automating the versioning process for JavaScript and CSS files (and is highly recommended).

Next we need to let Nginx know that the site is enabled, by symlinking it from sites-available to sites-enabled.
```
sudo ln -s /etc/nginx/sites-available/substanceis.com /etc/nginx/sites-enabled/substanceis.com
```
Finally you'll need to stop and start Nginx before Nginx knows that this virtual environment exists.
```
sudo /etc/init.d/nginx stop
sudo /etc/init.d/nginx start
```
And you're done configuring Nginx.

Now let's quickly create a default Django project.

su django
cd ~/domains/substanceis.com/
./substanceis.com/bin/python substanceis.com/bin/django-admin.py startproject hello
exit

Link the Django admin media into the public/media/ folder.

sudo ln -s /home/django/domains/substanceis.com/substanceis.com/lib/python2.5/site-packages/django/contrib/admin/media /home/django/domains/substanceis.com/public/media/admin

Then update the ADMIN_MEDIA_PREFIX setting in your hello/settings.py file

sudo emacs /home/django/domains/substanceis.com/hello/settings.py

to look like this

#ADMIN_MEDIA_PREFIX = '/media/'      
ADMIN_MEDIA_PREFIX = '/media/admin/'

Next let's create a Postgres database for this project to use.

sudo su postgres
createuser -P pg_substanceis
# should not be a superuser
# should not be able to create databases
# should not be able to create more new roles
createdb --encoding=UNICODE db_substanceis -O pg_substanceis
exit

Then we need to update the settings.py file with the proper settings.

sudo emacs hello/settings.py

And modify the DATABASE_* fields to look like this:

DATABASE_ENGINE = 'postgresql_psycopg2'
DATABASE_NAME = 'db_substanceis'
DATABASE_USER = 'pg_substanceis'                   
DATABASE_PASSWORD = '123456789'                   
DATABASE_HOST = ''                   
DATABASE_PORT = ''

Finally let's try to sync the database to verify that we got the settings correct.

~/domains/substanceis.com/substanceis.com/bin/python hello/manage.py syncdb

Assuming you entered your username, database and password correctly you should see something familiar.

Creating table auth_permission
Creating table auth_group
Creating table auth_user
Creating table auth_message
Creating table django_content_type
Creating table django_session
Creating table django_site
...

Our next step is to setup the project to memcached as its caching backend.
```
sudo emacs hello/settings.py
```
And add these settings:
```
CACHE_BACKEND =	'memcached://127.0.0.1:11211'
CACHE_MIDDLEWARE_SECONDS = 60 *	5
CACHE_MIDDLEWARE_KEY_PREFIX = 'sis' # SubstanceIS                  
CACHE_MIDDLEWARE_ANONYMOUS_ONLY = True
```
You'll have to decide what values of CACHE_MIDDLEWARE_SECONDS and CACHE_MIDDLEWARE_ANONYMOUS_ONLY are appropriate for the particular project you're configuring. There is more information available here.

The value of CACHE_MIDDLEWARE_KEY_PREFIX should be unique for each project using memcached.

After a tremendous journey, we reach the final step: configuring mod_wsgi and Apache to serve our project.

sudo emacs /home/django/domains/substanceis.com/hello/hello.wsgi

And add this code to hello.wsgi:

ALLDIRS = ['/home/django/domains/substanceis.com/substanceis.com/lib/python2.5/site-packages']
# note that the above directory depends on the locale of your virtualenv,
# and will thus be *different for each project!*
import os
import sys
import site

prev_sys_path = list(sys.path)

for directory in ALLDIRS:
    site.addsitedir(directory)

new_sys_path = []
for item in list(sys.path):
    if item not in prev_sys_path:
       new_sys_path.append(item)
       sys.path.remove(item)
sys.path[:0] = new_sys_path

# this will also be different for each project!
sys.path.append('/home/django/domains/substanceis.com/hello/')

os.environ['PYTHON_EGG_CACHE'] = '/home/django/.python-eggs'
os.environ['DJANGO_SETTINGS_MODULE'] = 'settings'
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

Next we need to create a virtual host for our domain.

sudo emacs /etc/apache2/sites-available/substanceis.com

And add this VirtualHost definition.

<VirtualHost 127.0.0.1:80>
    ServerName www.substanceis.com
    ServerAlias substanceis.com

    <Directory /home/django/domains/substanceis.com/hello/>
        Order deny,allow
        Allow from all
    </Directory>

    LogLevel warn
    ErrorLog  /home/django/domains/substanceis.com/log/apache_error.log
    CustomLog /home/django/domains/substanceis.com/log/apache_access.log combined

    WSGIDaemonProcess substanceis.com user=www-data group=www-data threads=25
    WSGIProcessGroup substanceis.com
    WSGIScriptAlias / /home/django/domains/substanceis.com/hello/hello.wsgi
</VirtualHost>

And finally enable substanceis.com.

sudo ln -s /etc/apache2/sites-available/substanceis.com /etc/apache2/sites-enabled/substanceis.com

And restart Apache2 to have it pick up the new site.

sudo apache2ctl graceful

You can repeat these steps to host as many projects as you want.

Some Additional Options

At this point your mod_wsgi server is setup and fully functional, but here are a couple of suggestions for things you might want to do with your fresh server.

At the moment if you go to your VPS's ip address, then you'll notice that it isn't serving anything at all. For most people that is probably the correct behavior, but you might want to serve a static page when they reach your VPS via its IP address (also, you can do the same thing to serve static content for some of your domains, rather than a full Django project).

First we need to create some folders
```
sudo mkdir /home/django/domains/default
sudo mkdir -p /home/django/domains/default/{public,logs}
```
Then create an index page
```
sudo emacs /home/django/domains/default/public/index.html
```
with this html
```
<html>
<head><title>Default</title></head>
<body><p>Hi. Welcome to my default site.</p></body>
</html>
```
Now we need to configure the Nginx site
```
sudo emacs /etc/nginx/sites-available/default
```
with these settings (you'll need to change 255.255.255.255 to your VPS's ip address):
```
server {

        listen   80;
        server_name 255.255.255.255;

        access_log /home/django/domains/default/log/access.log;
        error_log /home/django/domains/default/log/error.log;

        location / {
                 root /home/django/domains/default/public;
                 index index.html;
        }
```
Then symlink it into the sites-enabled folder.
```
sudo ln -s /etc/nginx/sites-available/default /etc/nginx/sites-enabled/default
```
And finally we need to stop-start nginx.
```
sudo /etc/init.d/nginx stop
sudo /etc/init.d/nginx start
```
Now if you navigate to your VPS's ip address in a browser, you'll see your static site. (Note that browsers typically limit the number of concurrent http connections to one domain, so you can speed up page load for heavy pages by using these steps to setup a media only subdomain--media.example.com, etc--for your Django sites and serving static media from them instead of the same domain as your site.)
One of the most helpful packages around is PIL, which supports a wide variety of image manipulation techniques. It's also used by some pieces of Django. It seems to be improperly packaged for installation with easy_install, but it is still straightforward to install.
```
su django
cd ~/libs
wget http://effbot.org/downloads/Imaging-1.1.6.tar.gz
tar -xzvf Imaging-*
cd Imaging-1.1.6
/domains/substanceis.com/substanceis.com/bin/python setup.py install
```
And now the substanceis.com virtual environment will have access to the Image module:
```
>>> import Image
>>> 
```
The more Django development you do, the more you'll be creating pluggable applications and reusing those made by others, so it's a good process to be familiar with. So let's take a few moments to install a couple common reusable Django applications developed by the Django community.

One of the downsides of using virtualenv is that you'll need to install pluggable apps into each of your environments one at a time. Of course, it also means that you can freeze a production site to use a certian version of a pluggable app while another plunges ahead into the unknown.

It is important to note the benefits of installing with setup.py or easy_install in conjunction with virtualenv. It means that you'll get a frozen copy of the code, and even if you update the source code later on, the environments won't be running different code.

On the other hand, if you're actively developing code, then you would be better off symlinking the code into the environment's site-packages directory so that you don't have to redeploy the code on changes.

Both approaches have their place, just be careful to use them appropriately.

First let's install django-compress, which I mentioned earlier and helps automating the merging and compression of JavaScript and CSS files.
```
su django
cd /django/home/libs
sudo svn checkout http://django-compress.googlecode.com/svn/trunk/ django-compress
cd django-compress
~/domains/substanceis.com/substanceis.com/bin/python setup.py install
```
Now let's make sure it's usable.
```
~/domains/substanceis.com/substanceis.com/bin/python
```
Then within Python try
```
>>> import compress
>>> 
```
As long as you don't get an error, then it installed correctly.

Next let's install django-faq, which is a pluggable application for creating faqs on websites (managable via the Django admin).

Just for fun, let's symlink it in (more convenient for active development) instead of using setup.py.
```
cd /django/home/libs
sudo git clone git://github.com/RockHoward/django-faq.git
cd django-faq
ln -s `pwd`/faq /home/django/domains/substanceis.com/substanceis.com/lib/python2.5/site-packages
```
Finally let's install django-mailer, which supplements Django with a more robust mailing framework.
```
cd /django/home/libs
sudo svn checkout http://django-mailer.googlecode.com/svn/trunk/ django-mailer
cd django-mailer
sudo ln -s `pwd`/mailer /home/django/domains/substanceis.com/substanceis.com/lib/python2.5/site-packages
```
Wait, is it okay to recommend one of my projects? Ok. You should consider taking a look at django-monetize, which supports dynamically serving different kinds of advertisements (and requests for donations, etc) based on a page's context.
```
cd /django/home/libs
sudo git clone git://github.com/lethain/django-monetize.git
cd django-monetize
sudo ln -s `pwd`/django_monetize /home/django/domains/substanceis.com/substanceis.com/lib/python2.5/site-packages
```
Moving past these pluggable apps, there are also a number of projects which have expanded to be something akin to mini-frameworks on top of Django. They provide a great amount of functionality, but often at the expense of somewhat non-standard installation patterns.

Two of the more popular are the social framework Pinax ( installation ), and the webshop platform Satchmo ( installation ).
Another thing I like doing is to use my VPS as a Git repository, which is--thanks to the manifold joys of DVCS--unexpectedly simple.
```
cd
mkdir git
mkdir git/scripts
cd git/scripts
echo "# a test" > test.py
git init --bare
```
Then exit your VPS and return to your machine. From your home machine you can clone that repository by using SSH.
```
mkdir my_repo
cd my_repo
git init
mkdir
echo "a test" > test.py
git add test.py
git commit -m "Initial commit."
git remote add origin ssh://67.207.149.179/home/will/git/scripts
git push origin master
```
Note that using the --bare keyword means that repository will keep all its files in its .git folder, so you won't be able directly interact with your files in that hosted repository. If you do want to interact with the files from that git repository on your machine, you should git clone from the origin repository and then push and pull like you normally would.

(If you're wanting to use Git as a mechanism for pushing code for your Django projects, you might want to read up about using something like Fabric.)
Although admittedly not as glamorous as being a Djangonaut, there are a number of situations where you may want Apache to also run some php scripts for you.
```
sudo apt-get install php5
sudo apache2ctl graceful
```
Then just place .php files in a directory served by Apache (not by Nginx), and they'll be executed. You'll probably want a combination of the Django and static Nginx configuration files that we've put together through the course of this walkthrough.

Resources

These articles draw heavily from my previous nginx+apache+mod_python+django and lighttpd+apache+mod_python+django articles from last year, but have taken advantage of new resources as well.

Many thanks to those who created these above resources that made this all possible.

Updated 2/13: Virtual Env

Tom Schreiber mentioned that virtualenv is a great tool for developing multiple projects with differing versions of libraries, and on his recommendation I updated the walkthrough to include it. Thanks Tom!

Updated, Again 2/13: Misc. Tweaks

A helpful Redditor pointed out some ways for improving. Particularly using chsh for changing shells (instead of manually editing the /etc/passwd file), and having Apache listen only to 127.0.0.1:80 instead of binding to *:8080, which makes it externally accessible, and having Nginx listen to external-ip:80 instead of *:80.

Hey kids, did you know that Nginx is pronounced "engine x". I sure as hell didn't. I actually had a coworker tell me how it was pronounced a while ago, but without seeing "engine x" written it never quite worked for me. Now I get it. Finally.↩

coulix reply

Golden ! Thanks a lot

tom schreiber reply Responding to coulix

Your older guides were heavily referenced on my initial slice setup, thanks for keeping them updated. Its a lot of little things to remember, guides are indispensable.

Recently began using virtualenv on slices with multiple projects, and that has been a real treat. Looking to try Fabric for deployment soon too.

Will Larson reply Responding to tom schreiber

Glad the guides are helpful. :) I haven't done a whole lot with virtualenv, but I hear great things. I'll take a look at it and see if I can integrate it into this guide.

shant reply

I'm getting ready to apploy my first Django app and this is just what I was looking for. Thank you very much.

One question, I have Ubuntu 8.04 installed in my slicehost, is there a reason to use Intrepid over that one?

Will Larson reply Responding to shant

It's possible some of the packages will be a bit different, but it shouldn't be a huge difference. Both use Python 2.5 (next Ubuntu release will have 2.6!), etc. I'm not sure if Hardy has Postgres 8.3, it might still be 8.2. That isn't a huge deal (this site is still running on my old instructions with Postgres 8.2 and mod_python... :/).

jr reply

Thanks! this is very useful!

Alex Ezell reply

This is a great overview. I just started rebuilding a slice yesterday, so this almanac found its way to me just in time.

Thanks for sharing your work and trials and tribulations!

florian reply

Hi,

some tips:

/usr/bin/editor is a symlink to /etc/alternatives/editor, which itself can be configured to set your favorite editor
(step 8/11) it's always a good idea to stay logged in until you're sure the newly created account can log in. Especially when changing ssh configuration etc. Usually, connections are not disconnected when restarting ssh. Should there be any error in your config, you won't be locked out of your system that way.
(step 10) ssh-copy-id saves some effort here
(step 12) it's git-core, not gitcore. libc6-dev is included in build-essential dependencies
(step 13) instead of sudo su postgres -c psql you can type sudo -u postgres psql
(step 14) instead of sudo python setup.py install maybe mention sudo checkinstall python set.py install here. This makes removal easier, as it ties into the package management
(step 15) you didn't install curl in your tutorial! wget is a dependency of ubuntu-standard, though.
(step 16) there's no package apache in intrepid. see apache2
(step 19) you can specify groups on chown like this: chown user:group filename

Will Larson reply Responding to florian

Hi Florian.

Thanks for your tips. I made most of the corrections you suggested. Honestly I'm not familiar with your tip #6 about

sudo checkinstall python set.py install

I'll have to look into that later.

Lenny reply

You gotta love the fact that someone would spend what was probably hours to create a guide for the sake of helping other people. Bravo!

shant reply

I get this error on the last line of step 18:

apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1 for ServerName [Sat Feb 14 18:10:15 2009] [warn] NameVirtualHost 127.0.0.1:80 has no VirtualHosts

Is that normal?

Also, when I did su django and made those dirs, at one point are we supposed to switch back to our user account because some of the next steps after that seem to require being back on our own user account. You might want to put that in the tutorial if that is the case.

shant reply

Sorry to post so many questions, but everything was fine up until I got to the 2nd step 4

sudo django-admin.py startproject hello

gives me an error saying there is no django-admin.py so I got around this by pointing to the bin folder that had that file in it. Should I be able to access django-admin.py without doing that?

Then in the next step, the symlink fails because there is no django folder in the /usr/lib/python2.5/site-packages/ folder. Did I screw up a step somewhere before?

shant reply

Forgot to mention, got this error after this command: sudo apache2ctl graceful

apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1 for ServerName [Sat Feb 14 19:04:31 2009] [warn] NameVirtualHost 127.0.0.1:80 has no VirtualHosts

I'm thinking this is where my problems are coming from.

Will Larson reply Responding to shant

Hi Shant,

That actually isn't an issue. It's just because the Apache virtual host is missing a domain specification, but that won't be causing anything to go wrong.

I cleaned up the article to reflect the errors you pointed out before. Basically the changed likes are like this:

 su django
cd ~/domains/substanceis.com/
./substanceis.com/bin/python substanceis.com/bin/django-admin.py startproject hello
exit
sudo ln -s /home/django/domains/substanceis.com/substanceis.com/lib/python2.5/site-packages/django/contrib/admin/media /home/django/domains/substanceis.com/public/media/admin

Thanks for working through this and giving me all this great feedback. It's pretty hard to avoid making some errors given the scope of the project and especially because I integrated Virtualenv into the buildout after I had already completed the original.

Will reply

How hard would it be to turn your process into a deb package? Is there anyone out there who is particularly gifted at such things and could do it very quickly with some cut-and-paste from this blog?

Thanks for the guide, this has already sped up my next deployment!

Damien reply

Hi,

About git, you should create a bare repository instead (git init|clone --bare) and have an other one for your deployed project. You something edit directly the files on your server and pushing some commit might make you lose your edits.

http://gitready.com/advanced/2009/02/01/push-to-only-bare-repositories.html

If you want to deploy your code using git, you can create a branch for you server, more stable than the master branch, and use a git hook (on your bare repository that you push to) to trigger the deployment of new code.

Thank for tutorial.

Will Larson reply Responding to Damien

Hi Damien,

Thanks for your suggestion, I've updated the article to reflect this suggestion. I never knew the --bare parameter existed...

Oleg reply

Hello!

Thanks for your article. Can somebody tell me why should i use apache with mod_wsgi? Is'n it better to use just ngnix with FCGI?

Will Larson reply Responding to Oleg

Hi Oleg,

I've actually written a piece schedule for tomorrow morning that discusses the tradeoffs between single-server and multi-server architecture at a high level.

The argument for using Apache and mod_wsgi versus Nginx and FCGI is some combination of performance, reliability, ease of installation, flexibility, and familiarity. For me personally this equation usually turns up Apache and mod_wsgi (although until recently mod_python would have been in its place).

That said, I do think there are compelling reasons for using multi-server architecture over single server, which tomorrow's article should explain

PS reply

Apparently I set up something wrong (probably in virtualenv, as I've never used it before) but when I try to python hello/manage.py syncdb it's blowing with an error on "No module named django.core.management"

What am I missing here? Probably something silly.

Will Larson reply Responding to PS

Obviously I haven't used virtualenv extensively myself, and backporting it into the tutorial has led to a series of mistakes that you are still rooting out for me and I am still fixing. Thanks for pointing them out to me, and sorry for making them!

After installing virtualenv, there shouldn't be any situations where you use sudo python or sudo easy_install. Instead you should always be using the virtualenv's Python or easy_install.

I think I've cleaned up these mistakes at this point. Hopefully. (Specifically using django_admin.py, manage.py, and installing cmemecache were all using the old sudo python terminology, but should now be fixed).

PS reply Responding to Will Larson

Yeah after the fact I found it was the wrong instance of python, which looks like what you updated the tutorial to indicate. Now I just need to get my VM set up with a real IP to properly test it.

Two thumbs up!

shant reply

Did something change in the setting up users or groups part of the tutorial? I've been through the tutorial a couple times and this time through I am having much more trouble with using my non-django user account to make folders inside the /home/django/ dir. I checked /etc/groups and I am part of the www-data group but I'm still getting a lot of Permission denied when trying to make folders inside /home/django/.

Alex Ezell reply Responding to shant

I'm having this same issue. My non-django account is also in the www-data group, yet can't write to anything under /home/django.

Le Roux reply

Why do you use such a complicated setup? Why apache? There are much better, faster, more efficient ways to deploy wsgi apps in my opinion and it doesn't necessarily mean "multi-server setup" (although it should more easily scale from 1 to many regardless)

I wrote some more about it here: http://www.reddit.com/r/Python/comments/7x8lb/a_healthy_environment_to_deploy_django_apache/c07oylh

..but I guess I'll have to blog about it all in detail to convince anyone. (Basically.. apache is horribly inefficient and slow probably 99% of the time and fastcgi is not the only other option either)

Will Larson reply Responding to Le Roux

In this regard I am most interested--only interested really--in numbers. There is so much disinformation floating out there about all of the servers and all configurations, that it takes a test (conducted with at least some regard to the scientific method and statistics) for me to really peak my interest.

This is what kept me away from mod_wsgi for over a year, and what keeps me with this flexible scalable multi-server approach as well.

That said I wrote about the single versus multi-server architecture this morning. I've been running some performance tests on nginx, apache & mod_wsgi versus apache & mod_wsgi for dynamic content in a configuration that strongly benefits the single-server setup, and will clean the data up at some point in the near future.

Carlo Zottmann reply

Excellent guide, thank you, it's very much appreciated.

There's a typo in step 11, sudo emacs /etc/ssh/shhd_config should read sudo emacs /etc/ssh/sshd_config.

Cheers, Carlo

Steve reply

Alternatively, get yourself a WebFaction ( http://www.webfaction.com ) and skip all these steps, everything is done for you (plus it's the best hosting you'll ever find)!

grant reply

About serving php with Apache...you already have Nginx on port 80, so do you just proxy to Apache by default, so php will work?

Thanks. Great post.

Billy reply

Hi, great tutorial, and timely for me too (you must get that alot).

Just a note, for me one of the first things that I have to install is

apt-get install language-pack-en

before installing anything else, otherwise I'll get alot of locale warnings and failures, and not being able to go very far.

Hambob reply

Having never touched mod_wsgi before i was going through parts of this step by step, following almost exactly what you have in terms of the hello.wsgi file and this seemed to trip me up: where you are appending the project dir to the path: sys.path.append('/home/django/domains/substanceis.com/hello/') i had to have this as: sys.path.append('/home/django/domains/substanceis.com/') before it would like it, otherwise i kept getting "ImportError: No module named hello" it seems to be working with the change, but am I missing something? or is this correct? On a side note. do you have any suggestions on setting this sort of thing up for multiple users(and insulating them from stomping on each others stuff)?

Will Larson reply Responding to Hambob

(I reposted the above comment after moving this blog from my mod_python slice to my mod_wsgi slice setup using this tutorial, so the formatting is my fault... sorry. ;)

Using the above setup the settings.py files should reference urls rather than hello.urls and so on, it has no knowledge the package name itself. This is relatively non-dangerous because each project is in its own virtualenv, so there won't be conflicts.
Virtualenvs are a big part of making a server multi-user safe. From where I'd make the Nginx and Apache configs for them and point them towards a folder within the user's folder instead of django user. That ought to be enough to keep them off each other's toes.

Rob reply

Wonderful, many thanks to you.

I something went slightly off when you changed to configure Apache2 to listen to 127.0.0.1:80. I'm getting an issue with NginX already listening to that port. The issue occurs because at the point in the article when you set up /etc/apache2/ports.conf you don't have any /etc/nginx/sites-enabled that are set to listen to only the external 255.255.255.255:80 ports, so I believe nginx defaults to listening to *:80. When I tried to set my apache to listen to 127.0.0.1:80 it said that port was already taken up. The solution is to set up nginx to only listen to the external IP's port 80 before starting apache listening to the internal port 80.

Cheers, Rob

feisal reply

Thanks for this!

john reply

Why do you put the django implementation into the

~django/domains/substanceis.com/substanceis.com

directory tree instead of in the

~django/domains/substanceis.com/

directory tree?

Billy reply

there's a small typo under '13', line 2 of the commands, it reads "psotgresql" instead of "postgresql".

cheers.

Coulix reply

Hi, On the wsgi conf

With project name foo os.environ['DJANGO_SETTINGS_MODULE'] = 'settings'

import foo.urls can not be found but import urls can.

Should it be os.environ['DJANGO_SETTINGS_MODULE'] = 'foo.settings' ?

Will Larson reply Responding to Coulix

Yes, it is possible that you end up needing foo.settings instead of simply settings. It depends a bit on exactly which paths are used for locating the folder containing the .wsgi file.

Giji reply

Hello i have done everything as said in guide but still there is an error:

Exception Value: No module named hello.urls

could you point some direction on this issue you can check full error message here http://dpaste.com/16315/

Thank you for this really good guide

nono reply

I wonder how you work with your repository setup and the way you made your file structure.

I'm using bazaar.

I think you need to do a lot to change something to your project.

Fergus Ross Ferrier reply

Extremely useful article which has helped me to set up my server.

A possible improvement: instead of setting up the virtualenv as a separate folder 'substanceis.com' inside another folder with that same name, you probably want to drop a level when you run virtualenv so that the bin / lib folders are created inside the topmost substanceis.com folder.

And if you then move into that substanceis.com folder, you can run ". bin/activate" which will automatically link python / easy_install to your virtual environment and mean you don't have to use lots of relative paths in your commands.

Billy reply

There are some differences between your hello.wsgi code with another sample .wsgi on this page: http://is.gd/qWh9 (search for "mysite.wsgi"), even though they look ALMOST exactly the same except for indentation levels.

But indentation level means alot in Python, who's code is correct?

rnd reply

Thank you very much for this insightful guide, I found it very helpful.

I also wanted to let you know that rather then using chsh for the users you create in the guide, you could just assign the group when you create the user as such:

useradd -m -s /bin/bash -U django

-m automatically creates the user's home directory.

-s sets the default shell used by the new user.

-U automatically creates a group by the same name as the username and adds the user to that group.

Wunki reply

First, thanks for this great post.

I just wanted to add a bit of information. If you want the real ip-address in your apache logs, instead of 127.0.0.1 than you should install the 'rpaf' module.

In debian:

sudo aptitude install libapache2-mod-rpaf

I think it would be something similar in Ubuntu.

onno reply

Why not put nginx on port 81 and put in your djano settings this. Then You don't need this proxy thing and your ip are clear.

Also, why use ngnix and not lighthttp

Thanxs for this almanac.

Ralph reply

Howdy, This was a nice idea for a page and I hope you revisit at some time. It is really just what I needed at the moment, but I'll move on and see if I can find something similar that works. I believe the problem is that you started with sometthing that worked and patched it without ever testing it again. There are lots of things that just don't work. I see some of them referred to in comments. Permissions problems in the domains directory always made me do things a little different than what it said here, for example. In the end, I have no working django and no real idea of what to do to fix it. I did pick up a few useful tricks and so it was worth reading and trying. Thank you for the effort, Ralph

Grant reply

This is one of the most comprehensive walk-throughs that I have seen. As a beginner I really appreciate it as it touches so many important aspects that are often left out in the explanations. Thanks'

kerridge reply

Hi I had a problem following this,

with the apache:

(98)Address already in use: make_sock: could not bind to address 127.0.0.1:80 no listening sockets available, shutting down

and had to to change nginx config:-

sudo vim /etc/nginx/sites-enabled/default

so that server was listening on external ip eg:

listen 192.168.8.244:80;

running through the rest now,thanks

jareksuaf reply

Hi thanks for sharing your knowledge I am following your excellent tutorial but have some problems at the following point

~$ sudo memcached -u www-data -p 11211 -m 32 -d bind(): Address already in use failed to listen

anonymous reply Responding to jareksuaf

Found the solution memcached was already started on my ubuntu server 9.04 I've just changed the settings in /etc/memcached.conf so do sudo nano /etc/memcached.conf sorry

sudo emacs /etc/memcached.conf ;-)

change user from nobody to www-data change memory settings from 64 to 32 and that's it

Save the file and restart memcached

sudo /etc/init.d/memcached restart

Regards

anonymous reply

Also step 10 for those who on win. Just generate SSH2-2 RSA using PuTTYgen save generated key as private without paraphrases open the key with notepad copy the whole 4 public lines. Now using PuTTY ssh as will to your server then in the home directory for the user " will" create directory .shh and inside create file authorized_keys from the top of that dcument make the space write: ssh- rsa space again and then paste the previously copied public key. Make it one long line and save it. make sure the .ssh is owned by will then set permissions to 0700 file should be 0600. Now shh from putty as will and use generated key so host name: your servers ip now under data auto -login username: will expand the shh on tree and under Auth leave all default and select the key file you have previously generated. For easy access to files on your server you may use Win SCP.

Regards Regards

anonymous reply

step 16

sudo emacs /etc/apache2/ports.confg

I think should b sudo emacs /etc/apache2/ports.conf

jareksuaf again and again reply

as i go with this tutorial i am encountering more probs on step 17 if i do su django then mkdir it will try to mkdir in wills home directory I did

su django cd mkdir domains mkdir libs mkdir .python-eggs exit is 25 dec 09 if i manage to set it up it will be a kind of christmas present :-)

Great Christmas to all

anonymous reply

last opst i didn't noticed Switch over to the /home/django folder another thing is that the libs already exist so mkdir libs not realy needed

anonymous reply

Now problems with subversion.. $ svn co http://code.djangoproject.com/svn/django/trunk/ django svn: OPTIONS of 'http://code.djangoproject.com/svn/django/trunk': could not connect to server (http://code.djangoproject.com)

Please Help

anonymous reply Responding to anonymous

It wasn't subversion stupid me My server provider has blacklisted the djangoproject.com but have found mirror here

http://bitbucket.org/mirror/django/src/tip/django/trunk/

tje reply

Hi there,

I've got a problem here.

If I open http://localhost I got welcome to nginx But if I open http://127.0.0.1 I got ImportError No module named hello.urls.

Please help me.

Thanks

Irrational Exuberance

The Django and Ubuntu Intrepid Almanac

Setting up your Server

Some Additional Options

Resources

Updated 2/13: Virtual Env

Updated, Again 2/13: Misc. Tweaks

Reply to this entry