Installing Graphite 0.9.8 on Debian

August 26, 2011

Here's the steps I took to install Graphite 0.9.8 on Debian.

First of all we need to install dependencies in a virtualenv:

# apt-get install libcairo2-dev
# apt-get install python-cairo
# exit
$ mkvirtualenv graphite
$ easy_install twisted
$ easy_install django
$ easy_install gunicorn

Next, we need to download the three parts of graphite - carbon, whisper and the web interface.

$ wget -N "http://launchpad.net/graphite/1.0/0.9.8/+download/graphite-web-0.9.8.tar.gz"
$ wget -N "http://launchpad.net/graphite/1.0/0.9.8/+download/carbon-0.9.8.tar.gz"
$ wget -N "http://launchpad.net/graphite/1.0/0.9.8/+download/whisper-0.9.8.tar.gz"
$ mkdir graphite
$ cd graphite
$ tar xzvf ../graphite-web-0.9.8.tar.gz
$ tar xzvf ../carbon-0.9.8.tar.gz
$ tar xzvf ../whisper-0.9.8.tar.gz

Now we're going to install all carbon and whisper:

$ su
# cd whisper-0.9.8
# python setup.py install
# cd ../carbon-0.9.8
# python setup.py install
# cd /opt/graphite/conf
# cp carbon.conf.example carbon.conf
# cp storage-schemas.conf.example storage-schemas.conf
# exit

Next we're going to use the check-dependencies.py script to make sure we've installed everything we need.

$ cd ~/graphite/graphite-web-0.9.8
(graphite)user@drumcoder:~/graphite/graphite-web-0.9.8$ python check-dependencies.py 
[WARNING] Unable to import the 'mod_python' module, do you have mod_python installed for python 2.6.6?
This means you will only be able to run graphite in the development server mode, which is not
recommended for production use.
[WARNING]
Unable to import the 'memcache' module, do you have python-memcached installed for python 2.6.6?
This feature is not required but greatly improves performance.

[WARNING]
Unable to import the 'ldap' module, do you have python-ldap installed for python 2.6.6?
Without python-ldap, you will not be able to use LDAP authentication in the graphite webapp.

[WARNING]
Unable to import the 'txamqp' module, this is required if you want to use AMQP.
Note that txamqp requires python 2.5 or greater.
All necessary dependencies are met.
4 optional dependencies not met. Please consider the warning messages before proceeding.

This is good enough for me.

Now we're going to install the web application:

# python setup.py install

This installs the graphite web app alongside everything else in /opt/graphite.

Running Django

I had a bit of a nightmare getting this all to run in gunicorn. Here's the debian init script I ended up using:

#! /bin/sh
### BEGIN INIT INFO
# Provides:          gunicorn-graphite
# Required-Start:    $remote_fs $syslog
# Required-Stop:     $remote_fs $syslog
# Should-Start:      $nginx
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: virtualenv + gunicorn + nginx debian init script
# Description:       virtualenv + gunicorn + nginx debian init script
### END INIT INFO

# Author: Nicolas Kuttler <hidden>
#
# Please remove the "Author" lines above and replace them
# with your own name if you copy and modify this script.
#
# Enable with update-rc.d gunicorn-example start 30 2 3 4 5 . stop 70 0 1 6 .
# (parameters might not be necessary, test)

# Do NOT "set -e"

PROJECT=/opt/graphite/webapp/graphite/
VIRTUALENV=/home/tjs/.virtualenvs/graphite
PORT=8002
LOGDIR=/var/log/gunicorn
# PATH should only include /usr/* if it runs after the mountnfs.sh script
PATH=/bin:/usr/bin
USER=tjs
GROUP=www-data
IP=localhost
WORKERS=1
# I am lazy and just call the init script gunicorn-project
NAME=graphite
DESC=$NAME
LOGFILE="$LOGDIR/$NAME.log"
PIDFILE="/tmp/gunicorn_$NAME.pid"
CMD="gunicorn_django --user=$USER --group=$GROUP --daemon --workers=$WORKERS --bind=$IP:$PORT --pid=$PIDFILE --name=$NAME --log-file=$LOGFILE --log-level=info -D settingslive.py"

# Load the VERBOSE setting and other rcS variables
. /lib/init/vars.sh

# Define LSB log_* functions.
# Depend on lsb-base (>= 3.2-14) to ensure that this file is present
# and status_of_proc is working.
. /lib/lsb/init-functions

#
# Function that starts the daemon/service
#
do_start() {
  # Return
  #   0 if daemon has been started
  #   1 if daemon was already running
  #   2 if daemon could not be started
  if [ -e $PIDFILE ]; then
    return 1
  fi
  cd $PROJECT
  . $VIRTUALENV/bin/activate
  cd $PROJECT
  $CMD
  if [ $? = 0 ]; then
    return 0
  else
    return 2
  fi
}

#
# Function that stops the daemon/service
#
do_stop() {
  # Return
  #   0 if daemon has been stopped
  #   1 if daemon was already stopped
  #   2 if daemon could not be stopped
  #   other if a failure occurred
  if [ -f $PIDFILE ]; then
    PID=`cat $PIDFILE`
    rm $PIDFILE
    kill -15 $PID
    if [ $? = 0 ]; then
      return 0
    else
      return 2
    fi
  else
    return 1
  fi
}

do_reload() {
  if [ -f $PIDFILE ]; then
    PID=`cat $PIDFILE`
    kill -HUP $PID
    return $?
  fi
  return 2
}

case "$1" in
  start)
  [ "$VERBOSE" != no ] && log_daemon_msg "Starting $DESC" "$NAME"
  do_start
  case "$?" in
    0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
    2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
  esac
  ;;
  stop)
  [ "$VERBOSE" != no ] && log_daemon_msg "Stopping $DESC" "$NAME"
  do_stop
  case "$?" in
    0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;;
    2) [ "$VERBOSE" != no ] && log_end_msg 1 ;;
  esac
  ;;
  restart)
  log_daemon_msg "Restarting $DESC" "$NAME"
 do_stop
  case "$?" in
    0|1)
    do_start
    case "$?" in
      0) log_end_msg 0 ;;
      1) log_end_msg 1 ;; # Old process is still running
      *) log_end_msg 1 ;; # Failed to start
    esac
    ;;
    *)
      # Failed to stop
    log_end_msg 1
    ;;
  esac
  ;;
  reload)
  log_daemon_msg "Reloading $DESC" "$NAME"
  do_reload
  case "$?" in
    0) log_end_msg 0 ;;
    *) log_end_msg 1 ;;
  esac
  ;;
  *)
  echo "Usage: $SCRIPTNAME {start|stop|restart|reload}" >&2
  exit 3
  ;;
esac

:

Install Diamond

Diamond, available from http://opensource.brightcove.com/project/Diamond/ is a project for gathering stats about the local machine and then passing them through to graphite. Here's how I installed it:

# apt-get install ant
# ant install
# apt-get install python-configobj
# apt-get install python-pysnmp2
# useradd diamond
# /etc/init.d/diamond start
# ps ax | grep diamond
20517 ?        Sl     0:00 /usr/bin/python /usr/lib/python2.6/dist-packages/diamond/server.py -v
20528 pts/7    S+     0:00 grep diamond
# update-rc.d diamond defaults

References