Nginx+Unicorn Configuration for Multi-App Servers

Platform as a service is getting more and more traction recently. However it doesn't mean that one should start developing his applications with Heroku or Engine Yard in mind from now on. Self-managed solutions, be it own dedicated servers or using cloud infrastructure (like Rackspace Cloud), are still valid choices in most situations.

At Monterail we're maintaining tens of web applications that are hosted in all of above-mentioned environments. We developed most of them, but there are also some legacy ones (even Rails 1.2 or PHP). So there is plenty of factors that affect hosting infrastructure, and basically each app need to be evaluated individually.

The key factors in such evaluation are expected traffic (and growth) and resource consumption by application workers and accompanying services/components (e.g. background jobs). In our case some applications are not big enough to justify being run on a dedicated machine, they can happily coexist with others on a single server.

In our setup we typically use combination of nginx and unicorn. Nginx acts as a reverse proxy for Rails (or more precisely Rack) applications which run on unicorn servers. This accomplishes several important goals and is far more superior to using passenger module (with either nginx or apache).

First of all it helps you isolate applications from each other and from nginx itself. With passenger module you run all apps on one version of Ruby interpreter, also you need to restart whole http server when changing it's configuration or configuration of any application it proxies to, which results in downtime because application workers are restarted as well.

By separating application workers from http server you take a different approach. Workers are managed by unicorn processes (which can be started using different Ruby versions, so each app can use different one), so you can reconfigure and restart both application and nginx independently. When nginx restarts it simply connects to already running and listening unicorn processes.

Unicorn is able to listen on Unix sockets which are faster than TCP ones and are better choice for our setup where both nginx and applications run on the same machine. Also it's OS kernel doing load balancing for us. Another cool feature is zero-downtime deployments, you can upgrade your application's code, unicorn library or even Ruby interpreter without dropping connections. Read more about unicorn internals in Ryan Tomayko's great article http://tomayko.com/writings/unicorn-is-unix.

So here's example configuration of Rails application running on unicorn with nginx as reverse proxy.

First, let's add unicorn gem to our app's Gemfile

gem 'unicorn'

Create unicorn config file in config/unicorn.rb. Here's rather minimal version, you'll find a well documented configuration example in Unicorn repo.

# The rule of thumb is to use 1 worker per processor core available,
# however since we'll be hosting many apps on this server,
# we need to take a less aggressive approach
worker_processes 2

# We deploy with capistrano, so "current" links to root dir of current release
working_directory "/var/www/appname/current"

# Listen on unix socket
listen "/tmp/unicorn.appname.sock", :backlog => 64

pid "/var/www/appname/current/tmp/pids/unicorn.pid"

stderr_path "/var/www/appname/current/log/unicorn.log"
stdout_path "/var/www/appname/current/log/unicorn.log"

You can start unicorn right away with

bundle exec unicorn -D -c /var/www/appname/current/config/unicorn.rb

Now the nginx configuration:

upstream unicorn-appname {
  server unix:/tmp/unicorn.appname.sock;
}

server {
  listen 80;
  server_name appname.com;
  root /var/www/appname/current/public;

  location / {
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Real-IP$remote_addr;
    proxy_set_header Host $http_host;
    proxy_redirect off;

    if (!-f $request_filename) {
      proxy_pass http://unicorn-appname;
      break;
    }
  }
}

Note the proxy_set_header directives. Nginx by default does not pass these down the line, so you would end up with your Rails app seeing your server's IP instead of remote IP address for example. Similarly you need to tell nginx to set X-FORWARDED-PROTO header if you want to detect HTTPS requests in you app:

server {
  listen 443;
  server_name appname.com;
  root /var/www/appname/current/public;

  location / {
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Real-IP$remote_addr;
    proxy_set_header Host $http_host;
    proxy_set_header X-FORWARDED-PROTO https;
    proxy_redirect off;

    if (!-f $request_filename) {
      proxy_pass http://unicorn-appname;
      break;
    }
  }
}

Restart nginx and your app should be up and running. We also want our app to start automatically after server boot, so we'll set up upstart script to achieve this.

#!/bin/sh
set -e

# Feel free to change any of the following variables for your app:
TIMEOUT=${TIMEOUT-60}
APP_ROOT=/var/www/appname/current
APP_USER=appname
PID=$APP_ROOT/tmp/pids/unicorn.pid
ENV=production
CMD="bundle exec unicorn_rails -E $ENV -D -c $APP_ROOT/config/unicorn.rb"
action="$1"
set -u

old_pid="$PID.oldbin"

cd $APP_ROOT || exit 1

sig () {
        test -s "$PID" && kill -$1 `cat $PID`
}

oldsig () {
        test -s $old_pid && kill -$1 `cat $old_pid`
}

case $action in
start)
        sig 0 && echo >&2 "Already running" && exit 0
        su --login $APP_USER -c "$CMD"
        ;;
stop)
        sig QUIT && exit 0
        echo >&2 "Not running"
        ;;
force-stop)
        sig TERM && exit 0

        echo >&2 "Not running"
        ;;
restart|reload)
        sig HUP && echo reloaded OK && exit 0
        echo >&2 "Couldn't reload, starting '$CMD' instead"
        su --login $APP_USER -c "$CMD"
        ;;
upgrade)
        if sig USR2 && sleep 2 && sig 0 && oldsig QUIT
        then
                n=$TIMEOUT
                while test -s $old_pid && test $n -ge 0
                do
                        printf '.' && sleep 1 && n=$(( $n - 1 ))
                done
                echo

                if test $n -lt 0 && test -s $old_pid
                then
                        echo >&2 "$old_pid still exists after $TIMEOUT seconds"
                        exit 1
                fi
                exit 0
        fi
        echo >&2 "Couldn't upgrade, starting '$CMD' instead"
        su --login $APP_USER -c "$CMD"
        ;;
reopen-logs)
        sig USR1
        ;;
*)
        echo >&2 "Usage: $0 "
        exit 1
        ;;
esac

Put this file in /etc/init.d/unicorn.appname and make it executable with

chmod +x /etc/init.d/unicorn.appname

The script above is a modified version of example from Unicorn repo. We changed it so each application is run under different user (note $APP_USER variable).

The last piece of the puzzle is monitoring the application and restarting it after new version is deployed. We use monit to watch tmp/restart.txt which is touched after each capistrano deployment.

check process unicorn-appname
  with pidfile /var/www/appname/current/tmp/pids/unicorn.pid
  start program = "/etc/init.d/unicorn.appname start"
  stop program = "/etc/init.d/unicorn.appname stop"
  group unicorn-appname

  check file appname-restart with path /var/www/appname/current/tmp/restart.txt
    if changed timestamp
    then exec "/usr/sbin/monit -g unicorn-appname restart"

Restart your monit process to reload it's configuration files and that's it. What I described above can be easily applied to multiple apps on the same server, you just need to change appname to your application's name.

Useful resources:

Michał Szajbe