CI, Jenkins and tests

I'm starting a new job soon. Or, rather, I'm starting my old job at a new employer. I'll be working on the same projects, the same software platform, just a better place. Among other things, this will give me the chance to introduce some new workflows and processes for the projects I'm working on - and in that light, setting up continuous integration seems a rather good idea.

One reason for that is that's just a good thing to do. In fact, it's among The Right Things To Do, as far as I'm aware. Reading Martin Fowler on the subject, Kent Beck (Test Driven Development) or Uncle Bob (The Clean Coder) and it won't be long before you're ready to get cracking. Another reason is that I won't be the only person working on the projects - part of the work will be done outside my new company. Given that, having a CI system setup, that will run all commits through the automated tests, letting me know what happened, just seems like the only sane thing to do.

So what follows is a recipe for setting up CI in the form of Jenkins, on a Debian 7 machine, running PHP projects through Gitolite.

Setting up Gitolite

Apt-get install gitolite gets you rolling. After installing the package, run dpkg-reconfigure gitolite and it'll let you set it up properly with a path to store repositories in, an SSH key for the admin user, and setting the user gitolite runs as.

This will install Gitolite 2.3 - if you require a newer version you should add the relevant versions in apt and pin the package.

When done, add the projects you need to manage to the config file, and add the proper users as well.

Setting up Jenkins

Jenkins is a Java app, so you'll need to install Java as the main dependency. The test server I am running on is as mentioned a Debian 7. It's only used as a server, no screen output. Hence, running Jenkins should preferably be headless, as no desktop is needed. This means installing openjdk-7-jre-headless with apt-get. After that, apt-get install jenkins.

This will give you an install of Jenkins managed through apt. An alternative to this is just downloading the Jenkins app directly from their site and running it manually, like java -jar jenkins.war.

If you went with the apt-managed version, you can adapt settings in /etc/default/jenkins - things like port and address to listen on, for instance. Most of the settings for Jenkins are handled through the web interface, though. You most likely want to specify which users can administrate Jenkins - add jobs, plugins, change settings, etc.

If you installed through apt you should also have a jenkins user now. Switch to this user, generate an SSH keypair using ssh-keygen, and stick the public key in gitolite with read access to the projects you want to use it for.

Setting up Jenkins plugins

To get Jenkins to integrate with Gitolite, you need a couple of plugins. I went with the following:

I added a bunch of others too, but haven't used them yet nor had the need to use them - they aren't needed to get the Jenkins+PHP+Gitolite combo set up.

Setup environment

Again, going with apt-get: apt-get install php5 phpunit. Also, add any/all php extensions you need. To check that you have everything you need, manually check out a copy of the project and try running your tests. If it works, you're ready to setup the build job in Jenkins.

Setting up first job

Go to the web interface for Jenkins and click on New Job in the dashboard. Give it a proper name and choose free-style software project. On the configuration page for the project, specify Source Code Management. Here you choose Git and give it the url for gitolite - if it's on the same server, it's likely something like gitolite@localhost:project-name.git. Choose one or more branches to checkout and build. After that, make sure you check the Poll SCM checkbox - the GIT plugin won't allow gitolite to kick off a build if that settings isn't checked.

Finally, add post-build actions as needed. Here, I'm just going with a shell script to kick off the tests - on fail, it'll exit with an exit code other than 1, which will fail the project build. I prefer to keep things simple - easier to manage. Hence a single, small shell script seems nice to me. There is a very good alternative maintained at http://jenkins-php.org/ - they have a recipe for setting up Jenkins to run PHP project builds. This includes details on plugins and software to install - it also includes a nice big ant script, that will run all tests and all sorts of other things. However, to me that means getting into ant to figure out how that works, plus taking things I don't need out of the script. When a shell script works just dandy, there's no reason to take on another technology, I find - I'm already learning four (4) other languages, so I don't need to pile on ant scripts.

When you've got the build job set up, try kicking off the build manually. If you got everything set up properly, you should see your project building fine (assuming you don't have any failing tests).

Automatic builds

The last bit to set up is the automatic building of projects. Here, you add a post commit hook to gitolite. I found lots of pages/hints on how to do this - none of them working, for some odd reason. What worked for me was setting up a common post-receive hook in /var/lib/gitolite/.gitolite/hooks/common/post-receive. The code looks like:

JENKINS_URL=https://localhost:8010
GIT_URL=gitolite@localhost
/usr/bin/touch /tmp/test
/usr/bin/wget -q $JENKINS_URL/git/notifyCommit\?url=$GIT_URL:$GL_REPO -O /dev/null

After setting this up, I then symlink it from the individual repositories (paths like /var/lib/gitolite/repositories/\<repo>/hooks/). Somehow, I couldn't find a proper description of how to do this - the documentation at the Gitolite page is rather poor and doesn't help in any way. Most of the descriptions for it seem to aim at Gitolite 3.

Anyway, what the hook does is make a request to Jenkins (remember to change the port in the above script to what you're running Jenkins on), specifically to an address provided by a Git plugin (this is why you need to make sure about having checked the Poll SCM thing), which will then kick off a build. You can of course also take other actions in this hook - but for Jenkins, nothing more is needed in order to start the build.