A tale of Pylons, Python and FastCGI on Dreamhost

micampe — Sun, 26 Nov 2006 08:01:00 -0800

Pylons is a WSGI based Python Web Framework that tries to combine the best ideas from other existing frameworks and does so by glueing togheter existing projects and allowing to swap its components at will and easily extend its capabilites with other WSGI components.

I will explain here how I installed it and a demo application on Dreamhost and how to avoid the single problem I had.

This procedure isn’t specific to either Dreamhost or Pylons, it can be used anywhere and for any WSGI compliant application if you don’t have root access to install new modules or if you want to keep the system wide installation clean and add modules just for a single user.

The installation procedure is quite easy and mostly automated, start by running these commands from your home directory on the remote shell:

mkdir bin
wget -P bin http://peak.telecommunity.com/dist/virtual-python.py
python2.4 bin/virtual-python

It’s important that you use python2.4 here, not just python.

This command will create some directories and symlinks on your home directory that will result in you little private Python environment, where you have the power to install new packages at will. To make the illusion more real, you should then add ~/bin in front of your $PATH, so that you can just type python in the shell and the interpreter used will be your private one instead of the global one.

export PATH="~/bin:$PATH"    # add this to ~/.bash_profile
wget -P bin http://peak.telecommunity.com/dist/ez_setup.py
python bin/ez_setup.py

This will install the easy_install script that you will use to install Pylons and other Python modules you can find on the Cheese Shop:

easy_install Pylons

This will install the Pylons framework and all its dependencies. To test the installation we will use the QuickWiki demo application:

easy_install QuickWiki

This will install the application and some more dependencies. The next step is creating the configuration file for the application:

paster make-config QuickWiki wiki.ini

In this file you only have to change the database URI, for this test we will use SQLite:

sqlalchemy.dburi = sqlite:///quickwiki.db

And setup the application to create the database:

paster setup-app wiki.ini

Then we need a script to start the application. Pylons, as every WSGI application can be deployed in a number of different ways, but the only one you can use on Dreamhost is FastCGI. At this point, every tutorial around the web will tell you to use flup and then there will be pain and hair pulling because you won’t understand why it’s not working (unless you ask Allan Saddi to check your setup, that is. Thank you Allan!).

The solution is to use the supposed to be obsolete fcgi.py script, so let’s install it:

wget -P lib/python2.4/site-packages http://svn.saddi.com/py-lib/trunk/fcgi.py

We said we needed a script, so put this in a file named server.fcgi in your web directory and make it executable:

#!/home/username/bin/python

from paste.deploy import loadapp
from fcgi import WSGIServer

app = loadapp('config:/home/username/www.yourdomain.net/wiki.ini')
server = WSGIServer(app)
server.run()

You will need to correct the paths here, as we need them to be absolute.

Now when you point at this file in a web browser you should see the QuickWiki FrontPage.

The problem with flup (as found by Allan itself) is that it takes too long to start and it seems that FastCGI on Dreamhost will wait for a maximum of 3 seconds. fcgi.py instead is simpler (doesn’t do thread pooling for example) and therefore faster to start. This means that when the server is under heavy load there could still be problems, but it has been pretty reliable.

The best solution to this problem would be SCGI support by Dreamhost, if you already are a customer, go vote for it!

Michele Campeotto: Tag pylons

A tale of Pylons, Python and FastCGI on Dreamhost