Buildbot
From charlesreid1
Buildbot creates a framework for automated code testing.
Installation
Tarballs can be obtained from here: http://trac.buildbot.net/
Download/installation instructions are here: http://trac.buildbot.net/wiki/DownloadInstall
Since I was building Buildbot on a system where I had no administrative permissions, I used virtualenv (see Python#Virtual Python: virtualenv for more information).
I downloaded the master and slave tarballs, extracted them, and ran:
$ ptyhon setyp.py install --prefix=/path/to/virtualenv
After doing this, I can use the python binary in /path/to/virtualenv/bin/python to load the buildbot library through python.
Once setup.py is finished, I ran:
$ buildbot --version Buildbot version: 0.8.3p1 Twisted version: 11.0.0 $ buildslave --version Buildslave version: 0.8.3 Twisted version: 11.0.0
Mock Package
In order to run BuildBot, you will also need the Mock package, which is also related to testing.
Obtain Mock here: http://pypi.python.org/pypi/mock#downloads
Creating Buildbots
Unit Test Suite
export PYTHONPATH=".:${PYTHONPATH}"; trial buildbot.test
export PYTHONPATH=".:${PYTHONPATH}"; trial buildslave.test
(Note that although you can build Buildbot without Mock, the above test suites will fail without mock.)
For me, the slave test fails with:
$ export PYTHONPATH=".:${PYTHONPATH}"; trial buildslave.test
buildslave
test ... [ERROR]
===============================================================================
[ERROR]
Traceback (most recent call last):
File "/home/u0552682/pkg/virtualenv/lib/python2.4/site-packages/Twisted-11.0.0-py2.4-linux-x86_64.egg/twisted/trial/runner.py", line 677, in loadByNames
things.append(self.findByName(name))
File "/home/u0552682/pkg/virtualenv/lib/python2.4/site-packages/Twisted-11.0.0-py2.4-linux-x86_64.egg/twisted/trial/runner.py", line 487, in findByName
return reflect.namedAny(name)
File "/home/u0552682/pkg/virtualenv/lib/python2.4/site-packages/Twisted-11.0.0-py2.4-linux-x86_64.egg/twisted/python/reflect.py", line 475, in namedAny
obj = getattr(obj, n)
exceptions.AttributeError: 'module' object has no attribute 'test'
buildslave.test
Creating a Buildmaster
Documentation on creating buildmaster: http://buildbot.net/buildbot/docs/current/Creating-a-buildmaster.html#Creating-a-buildmaster
I created a buildmaster like this:
$ buildbot create-master -r experiment_master
This creates master.cfg.sample, which is a sample configuration file
Creating a Buildslave
Documentation on creating buildslave: http://buildbot.net/buildbot/docs/current/Creating-a-buildslave.html#Creating-a-buildslave
I created a buildslave like this:
$ buildslave create-slave project_directory host:port slavename password
Buildslave creation options: http://buildbot.net/buildbot/docs/current/Buildslave-Options.html#Buildslave-Options
I'm still not clear on what the hostname:port significance is... like, what if the master and slave are on the same machine?
Running Buildbots
Daemon
You can stat the buildmaster daemon by running:
buildbot start /path/to/buildmaster/directory
and likewise, yo ucan start the buildslave daemon by running:
buildslave start /path/to/buildslave/directory
Errors?
I'm seeing errors like:
$ buildbot start experiment_master
Following twistd.log until startup finished..
Removing stale pidfile /home/u0552682/pkg/buildbot/experiment_master/twistd.pid
2011-05-11 20:08:03-0600 [-] Log opened.
2011-05-11 20:08:03-0600 [-] twistd 11.0.0 (/usr/bin/python 2.4.3) starting up.
2011-05-11 20:08:03-0600 [-] reactor class: twisted.internet.selectreactor.SelectReactor.
2011-05-11 20:08:03-0600 [-] Creating BuildMaster -- buildbot.version: 0.8.3p1
2011-05-11 20:08:03-0600 [-] loading configuration from /home/u0552682/pkg/buildbot/experiment_master/master.cfg
2011-05-11 20:08:03-0600 [-] unable to import dnotify, so Maildir will use polling instead
2011-05-11 20:08:03-0600 [-] creating adbapi pool: pysqlite2.dbapi2 ('/home/u0552682/pkg/buildbot/experiment_master/state.sqlite',) {'check_same_thread': False, 'cp_noisy': True, 'cp_reconnect': True}
2011-05-11 20:08:03-0600 [-] twisted.spread.pb.PBServerFactory starting on 8099
2011-05-11 20:08:03-0600 [-] Starting factory <twisted.spread.pb.PBServerFactory instance at 0x2ab7da8ab248>
2011-05-11 20:08:03-0600 [-] adding new builder runtests for category None
2011-05-11 20:08:03-0600 [-] trying to load status pickle from /home/u0552682/pkg/buildbot/experiment_master/runtests/builder
2011-05-11 20:08:03-0600 [-] /home/u0552682/pkg/virtualenv/lib/python2.4/site-packages/Twisted-11.0.0-py2.4-linux-x86_64.egg/twisted/persisted/styles.py:160: exceptions.DeprecationWarning: twisted.python.reflect.allYourBase was deprecated in Twisted 11.0.0; please use inspect.getmro instead
2011-05-11 20:08:03-0600 [-] added builder runtests in category None
2011-05-11 20:08:03-0600 [-] setBuilders._add: [<buildbot.util.loop.DelegateLoop instance at 0x2ab7d9fa4908>, <BuildSlave 'slavename', current builders: >] [<Builder ''runtests'' at 46969133905824>]
2011-05-11 20:08:03-0600 [-] adding IStatusReceiver <WebStatus on port tcp:8099 at 0x2ab7da312248>
2011-05-11 20:08:03-0600 [-] configuration update started
2011-05-11 20:08:03-0600 [-] Unhandled Error
Traceback (most recent call last):
File "/home/u0552682/pkg/virtualenv/lib/python2.4/site-packages/Twisted-11.0.0-py2.4-linux-x86_64.egg/twisted/application/service.py", line 184, in setServiceParent
self.parent.addService(self)
File "/home/u0552682/pkg/virtualenv/lib/python2.4/site-packages/Twisted-11.0.0-py2.4-linux-x86_64.egg/twisted/application/service.py", line 303, in addService
service.privilegedStartService()
File "/home/u0552682/pkg/virtualenv/lib/python2.4/site-packages/Twisted-11.0.0-py2.4-linux-x86_64.egg/twisted/application/internet.py", line 357, in privilegedStartService
self._waitingForPort = self.endpoint.listen(self.factory)
File "/home/u0552682/pkg/virtualenv/lib/python2.4/site-packages/Twisted-11.0.0-py2.4-linux-x86_64.egg/twisted/internet/endpoints.py", line 187, in listen
interface=self._interface)
--- <exception caught here> ---
File "/home/u0552682/pkg/virtualenv/lib/python2.4/site-packages/Twisted-11.0.0-py2.4-linux-x86_64.egg/twisted/internet/defer.py", line 104, in execute
result = callable(*args, **kw)
File "/home/u0552682/pkg/virtualenv/lib/python2.4/site-packages/Twisted-11.0.0-py2.4-linux-x86_64.egg/twisted/internet/posixbase.py", line 419, in listenTCP
p.startListening()
File "/home/u0552682/pkg/virtualenv/lib/python2.4/site-packages/Twisted-11.0.0-py2.4-linux-x86_64.egg/twisted/internet/tcp.py", line 867, in startListening
raise CannotListenError, (self.interface, self.port, le)
twisted.internet.error.CannotListenError: Couldn't listen on any:8099: (98, 'Address already in use').
The buildmaster took more than 10 seconds to start, so we were unable to
confirm that it started correctly. Please 'tail twistd.log' and look for a
line that says 'configuration update complete' to verify correct startup.
It looks like it started OK: running netstat -tulpn shows that there is a python process using port 8099, and the twistd.log and twistd.pid files both exist. The pid using port 8099 matches the pid in the twistd.pid file.
Further, running ps ax | grep python shows the buildbot start process running...???
Configuring Buildbots
Master Configuration
Documentation on configuration files: http://buildbot.net/buildbot/docs/current/Configuration.html#Configuration
<graphviz> digraph G { "Master configuration File" -> "Scheduler"
"Scheduler" -> "Changes" "Scheduler" -> "Periodic Scheduler" -> "Periodic Builds" -> "BuildSet" "Scheduler" -> "BuildSet" "BuildSet" -> "BuildMaster" "BuildMaster" -> "BuildRequest1" -> "Builder1" "BuildMaster" -> "BuildRequest2" -> "Builder2" "BuildMaster" -> "BuildRequest3" -> "Builder3" "BuildMaster" -> "BuildRequest4" -> "Builder4"
} </graphviz>
Slave Configuration
(link here?)