Buildbot

Buildbot creates a framework for automated code testing.

Installation

Tarballs can be obtained from here: http://trac.buildbot.net/

Download/installation instructions are here: http://trac.buildbot.net/wiki/DownloadInstall

Prerequisites

VirtualEnv

Since I was building Buildbot on a system where I had no administrative permissions, I used virtualenv (see Python#Virtual Python: virtualenv for more information).

I downloaded the master and slave tarballs, extracted them, and ran:

$ ptyhon setyp.py install --prefix=/path/to/virtualenv

After doing this, I can use the python binary in /path/to/virtualenv/bin/python to load the buildbot library through python.

Mock

In order to run BuildBot, you will also need the Mock package, which is also related to testing.

Obtain Mock here: http://pypi.python.org/pypi/mock#downloads

Installing Buildbot

I downloaded Buildbot (both the master and the slave) from http://trac.buildbot.net/wiki/DownloadInstall

Master

I installed the Buildbot master by running

$ python setup.py build

The next thing I had to do was add the future installation directory to my $PYTHONPATH variable (otherwise the python installer throws an error). I put this in one of my Dot files (.bashrc):

export PYTHONPATH="${HOME}/pkg/buildbot/0.8.3_master/lib/python2.7/site-packages:${PYTHONPATH}"

then made the directory:

$ mkdir -p ${HOME}/pkg/buildbot/0.8.3_master/lib/python2.7/site-packages

and then ran setup's installer:

$ python setup.py install --prefix=${HOME}/pkg/buildbot/0.8.3_master

Slave

I basically repeated the master build process. First, I downloaded the tarball, then untarred it.

Then I ran:

$ python setup.py build

Then I added the path to the Python package to my $PYTHONPATH variable:

export PYTHONPATH="${HOME}/pkg/buildbot/0.8.3_slave/lib/python2.7/site-packages:${PYTHONPATH}"

then made the directory:

$ mkdir -p ${HOME}/pkg/buildbot/0.8.3_slave/lib/python2.7/site-packages

and then ran setup's installer:

$ python setup.py install --prefix=${HOME}/pkg/buildbot/0.8.3_slave

Confirming Installation

Once both installs finished, I added both to my path:

export PATH="${HOME}/pkg/buildbot/0.8.3_master/bin:${PATH}"
export PATH="${HOME}/pkg/buildbot/0.8.3_slave/bin:${PATH}"

and then ran:

$ buildbot --version
Buildbot version: 0.8.3p1
Twisted version: 11.0.0

$ buildslave --version
Buildslave version: 0.8.3
Twisted version: 11.0.0

Creating Buildbots

Unit Test Suite

export PYTHONPATH=".:${PYTHONPATH}"; trial buildbot.test

export PYTHONPATH=".:${PYTHONPATH}"; trial buildslave.test

(Note that although you can build Buildbot without Mock, the above test suites will fail without mock.)

For me, the slave test fails with:

$ export PYTHONPATH=".:${PYTHONPATH}"; trial buildslave.test
buildslave
  test ...                                                              [ERROR]

===============================================================================
[ERROR]
Traceback (most recent call last):
  File "/home/u0552682/pkg/virtualenv/lib/python2.4/site-packages/
         Twisted-11.0.0-py2.4-linux-x86_64.egg/twisted/trial/runner.py", line 677, in loadByNames
    things.append(self.findByName(name))
  File "/home/u0552682/pkg/virtualenv/lib/python2.4/site-packages/
         Twisted-11.0.0-py2.4-linux-x86_64.egg/twisted/trial/runner.py", line 487, in findByName
    return reflect.namedAny(name)
  File "/home/u0552682/pkg/virtualenv/lib/python2.4/site-packages/
         Twisted-11.0.0-py2.4-linux-x86_64.egg/twisted/python/reflect.py", line 475, in namedAny
    obj = getattr(obj, n)
exceptions.AttributeError: 'module' object has no attribute 'test'

buildslave.test

Creating a Buildmaster

Documentation on creating buildmaster: http://buildbot.net/buildbot/docs/current/Creating-a-buildmaster.html#Creating-a-buildmaster

I created a buildmaster like this:

$ buildbot create-master -r experiment_master

This creates master.cfg.sample, which is a sample configuration file

Creating a Buildslave

Documentation on creating buildslave: http://buildbot.net/buildbot/docs/current/Creating-a-buildslave.html#Creating-a-buildslave

I created a buildslave like this:

$ buildslave create-slave project_directory host:port slavename password

Buildslave creation options: http://buildbot.net/buildbot/docs/current/Buildslave-Options.html#Buildslave-Options

I'm still not clear on what the hostname:port significance is... like, what if the master and slave are on the same machine?

Running Buildbots

Daemon

You can stat the buildmaster daemon by running:

buildbot start /path/to/buildmaster/directory

and likewise, yo ucan start the buildslave daemon by running:

buildslave start /path/to/buildslave/directory

Errors?

I'm seeing errors like:

 $ buildbot start experiment_master
Following twistd.log until startup finished..
Removing stale pidfile /home/u0552682/pkg/buildbot/experiment_master/twistd.pid
2011-05-11 20:08:03-0600 [-] Log opened.
2011-05-11 20:08:03-0600 [-] twistd 11.0.0 (/usr/bin/python 2.4.3) starting up.
2011-05-11 20:08:03-0600 [-] reactor class: twisted.internet.selectreactor.SelectReactor.
2011-05-11 20:08:03-0600 [-] Creating BuildMaster -- buildbot.version: 0.8.3p1
2011-05-11 20:08:03-0600 [-] loading configuration from /home/u0552682/pkg/buildbot/experiment_master/master.cfg
2011-05-11 20:08:03-0600 [-] unable to import dnotify, so Maildir will use polling instead
2011-05-11 20:08:03-0600 [-] creating adbapi pool: pysqlite2.dbapi2 ('/home/u0552682/pkg/buildbot
                             /experiment_master/state.sqlite',) {'check_same_thread': False, 'cp_noisy': 
                             True, 'cp_reconnect': True}
2011-05-11 20:08:03-0600 [-] twisted.spread.pb.PBServerFactory starting on 8099
2011-05-11 20:08:03-0600 [-] Starting factory <twisted.spread.pb.PBServerFactory instance at 0x2ab7da8ab248>
2011-05-11 20:08:03-0600 [-] adding new builder runtests for category None
2011-05-11 20:08:03-0600 [-] trying to load status pickle from /home/u0552682/pkg/buildbot/
                             experiment_master/runtests/builder
2011-05-11 20:08:03-0600 [-] /home/u0552682/pkg/virtualenv/lib/python2.4/site-packages/
                             Twisted-11.0.0-py2.4-linux-x86_64.egg/twisted/persisted/styles.py:160: 
                             exceptions.DeprecationWarning: twisted.python.reflect.allYourBase was deprecated in 
                             Twisted 11.0.0; please use inspect.getmro instead
2011-05-11 20:08:03-0600 [-] added builder runtests in category None
2011-05-11 20:08:03-0600 [-] setBuilders._add: [<buildbot.util.loop.DelegateLoop instance at 0x2ab7d9fa4908>, 
                             <BuildSlave 'slavename', current builders: >] [<Builder ''runtests'' at 46969133905824>]
2011-05-11 20:08:03-0600 [-] adding IStatusReceiver <WebStatus on port tcp:8099 at 0x2ab7da312248>
2011-05-11 20:08:03-0600 [-] configuration update started
2011-05-11 20:08:03-0600 [-] Unhandled Error
	Traceback (most recent call last):
	  File "/home/u0552682/pkg/virtualenv/lib/python2.4/site-packages/Twisted-11.0.0-py2.4-linux-x86_64.egg
                /twisted/application/service.py", line 184, in setServiceParent
	    self.parent.addService(self)
	  File "/home/u0552682/pkg/virtualenv/lib/python2.4/site-packages/Twisted-11.0.0-py2.4-linux-x86_64.egg
                 /twisted/application/service.py", line 303, in addService
	    service.privilegedStartService()
	  File "/home/u0552682/pkg/virtualenv/lib/python2.4/site-packages/Twisted-11.0.0-py2.4-linux-x86_64.egg
                 /twisted/application/internet.py", line 357, in privilegedStartService
	    self._waitingForPort = self.endpoint.listen(self.factory)
	  File "/home/u0552682/pkg/virtualenv/lib/python2.4/site-packages/Twisted-11.0.0-py2.4-linux-x86_64.egg
                 /twisted/internet/endpoints.py", line 187, in listen
	    interface=self._interface)
	--- <exception caught here> ---
	  File "/home/u0552682/pkg/virtualenv/lib/python2.4/site-packages/Twisted-11.0.0-py2.4-linux-x86_64.egg
                 /twisted/internet/defer.py", line 104, in execute
	    result = callable(*args, **kw)
	  File "/home/u0552682/pkg/virtualenv/lib/python2.4/site-packages/Twisted-11.0.0-py2.4-linux-x86_64.egg
                 /twisted/internet/posixbase.py", line 419, in listenTCP
	    p.startListening()
	  File "/home/u0552682/pkg/virtualenv/lib/python2.4/site-packages/Twisted-11.0.0-py2.4-linux-x86_64.egg
                 /twisted/internet/tcp.py", line 867, in startListening
	    raise CannotListenError, (self.interface, self.port, le)
	twisted.internet.error.CannotListenError: Couldn't listen on any:8099: (98, 'Address already in use').
	

The buildmaster took more than 10 seconds to start, so we were unable to
confirm that it started correctly. Please 'tail twistd.log' and look for a
line that says 'configuration update complete' to verify correct startup.

It looks like it started OK: running netstat -tulpn shows that there is a python process using port 8099, and the twistd.log and twistd.pid files both exist. The pid using port 8099 matches the pid in the twistd.pid file.

Further, running ps ax | grep python shows the buildbot start process running...???

Configuring Buildbots

Conceptualization

Documentation on configuration files: http://buildbot.net/buildbot/docs/current/Configuration.html#Configuration

<graphviz> digraph G { "Master Configuration File" -> "Scheduler" "Master Configuration File" -> "Periodic Scheduler"

 "Scheduler" -> "Changes"
 "Periodic Scheduler" -> "Changes"

 "Periodic Scheduler" -> "Periodic Builds"
   "Periodic Builds" -> "BuildSet"

 "Scheduler" -> "BuildSet"

   "BuildSet" -> "BuildMaster"

     "BuildMaster" -> "BuildRequest1" -> "Associated BuildSlaves" 
     "BuildMaster" -> "BuildRequest2" -> "Associated BuildSlaves"
     "BuildMaster" -> "BuildRequest3" -> "Associated BuildSlaves"
     "BuildMaster" -> "BuildRequest4" -> "Associated BuildSlaves"

       "Associated BuildSlaves" -> "Builder1" -> "BuildFactory1" -> "Build1"
       "Associated BuildSlaves" -> "Builder2" -> "BuildFactory2" -> "Build2"
       "Associated BuildSlaves" -> "Builder3" -> "BuildFactory3" -> "Build3"
       "Associated BuildSlaves" -> "Builder4" -> "BuildFactory4" -> "Build4"

} </graphviz>

In words:

The BuildMaster has Scheduler objects. These scheduler objects create BuildSet objects and give them to the BuildMaster. The BuildMaster then determines how to construct BuildRequests out of the BuildSet objects, and passes the BuildRequests to individual Builders (or BuildSlaves).

Each Builder shares a common operating system, libraries, compilers, headers, etc. If any of these are different, there should be a different builder.

A Builder can be part of multiple BuildSlaves. This is for redundancy; e.g. if there is a single Builder that is part of two BuildSlaves, and one of them goes offline, the Builder still receives instructions from the BuildSlave that has not gone offline.

A Build is a single compilation or test run of a particular version of code. (This is typically something like a checkout, configure, make, make check sequence).

The Builder uses a BuildFactory to specify the steps involved in the Build.