Metadata-Version: 1.0
Name: collective.buildbot
Version: 0.2.0
Summary: A set of zc.buildout recipes and package to easy install buildbot
Home-page: UNKNOWN
Author: Gael Pasgrimaud
Author-email: gael.pasgrimaud@ingeniweb.com
License: ZPL
Description: .. contents::
        
        .. - Code repository: https://svn.plone.org/svn/collective/collective.buildbot
        .. - Questions and comments to support_at_ingeniweb.com
        .. - Report bugs at http://trac.ingeniweb.com
        
        
        Change history
        **************
        
        0.2.0 (2008-05-21)
        ==================
        
        - Added irc options so you can attach an irc bot to the master buildbot
        [mustapha]
        
        - Allow public_html customization [gawel]
        
        - Added custom about page to link to collective.buildout [gawel]
        
        - Added support for Git repositories [dokai]
        
        - Refactored the repository URL configuration. For Subversion, you
        should use only the ``repository`` option to specify a full URL to
        the desired branch (trunk, tag or branch) that will be built. For
        Git in addition to setting the ``repository`` option you can use
        the ``branch`` option to specify a specific branch to build. By
        default the ``master`` branch will be used for Git
        repositories. [dokai]
        
        - Cleaned up a lot of redundant imports. [dokai]
        
        - Updated the documentation and examples. [dokai]
        
        - Deprecated the collective.buildbot:projects recipe [dokai]
        
        - Fixed problem with missing twistd.log files on first run [dokai]
        
        - Fixed bug that prevented the master from starting if there weren't
        any SVN pollers configured. [dokai]
        
        - Added new options ``periodic-scheduler`` and ``cron-scheduler`` to
        set up passive schedulers for projects. [dokai]
        
        
        0.1.1 (2008-05-02)
        ==================
        
        - bugs fixes [gawel]
        
        0.1.0 (xxxx-xx-xx)
        ==================
        
        - Created recipe with ZopeSkel [Gael Pasgrimaud].
        
        Package description
        *******************
        
        This package provides a set of zc.buildout_ recipes that make it easy
        to configure a buildbot_ setup (build master, build slaves and
        projects) and a scripts to run the buildbot master and slaves. The
        recipes produce INI-style declarative configuration files based on the
        buildout configuration. These configuration files are in turn read by
        the buildbot runner script to initialize the buildbot environment.
        
        The available recipes are:
        
        * ``collective.buildbot:master`` -- Produces a configuration file
        for the build master process.
        
        * ``collective.buildbot:slave`` -- Produces a configuration file for
        the build slave process.
        
        * ``collective.buildbot:project`` -- Produces a configuration for a
        project build on a selected slave.
        
        * ``collective.buildbot:poller`` -- Produces configuration for code
        repository pollers.
        
        * ``collective.buildbot:pollers`` -- Produces configuration for code
        repository pollers by allowing multiple pollers to be configured
        in a single buildout section.
        
        It is possible to use all the recipes in a single buildout and have
        both the master and slave(s) on the same machine. However, in most
        cases you will have one buildout for the build master that uses the
        ``collective.buildbot:master`` and ``collective.buildbot:project`` to
        set up the build processes and then separate buildouts on each of the
        slave machines that use the ``collective.buildbot:slave`` recipe.
        
        .. _zc.buildout: http://pypi.python.org/pypi/zc.buildout
        .. _buildbot: http://pypi.python.org/pypi/buildbot
        
        
        The build master recipe
        ***********************
        
        The ``collective.buildbot:master`` recipe produces a configuration
        file that sets up the build master process. Once the build master is
        configured you can run in by executing the controller script under the
        buildout's bin directory. The controller script will be named after
        the section name, so if you had a ``[buildmaster]`` section in your
        buildout.cfg you would get a ``bin/buildmaster`` script.
        
        Supported options
        =================
        
        The recipe supports the following options:
        
        ``port``
        The port the build master process listens for connections from
        build slaves. The slaves must be configured to use the
        corresponding port in the sections using the
        ``collective.buildbot:slave`` recipe.
        
        ``wport``
        The web port for serving the buildbot web interface.
        
        ``project-name``
        Project name. Displayed on the web interface.
        
        ``project-url``
        Project url, used on the web interface.
        
        ``url``
        buildbot url.
        
        ``build-slaves``
        A sequence of build slave configurations. Each build slave must be
        defined on a separate line containing the name of the build slave
        and the password for the build slave separated by white space.
        
        ``allow-force`` (optional)
        If ``true`` allows users to force builds using the web
        interface. Defaults to ``false``.
        
        ``public-html`` (optional)
        Location of a directory that contains custom resources (HTML, CSS,
        images) for the web interface.
        
        
        Additionally you can use the following options if you need to run an
        IRC bot:
        
        ``irc-host``
        The irc host to connect to. ie: irc.freenode.net
        
        ``irc-channels``
        A list of channels to join. #plone
        
        ``irc-nickname``
        The bot nickname. Defaults to ``buildbot``
        
        ``irc-password``
        The password used to identify the bot. Defaults to an empty string
        
        Example usage
        =============
        
        We'll start by creating a buildout that uses the recipe::
        
        >>> write('buildout.cfg',
        ... """
        ... [buildout]
        ... parts = buildmaster
        ...
        ... [buildmaster]
        ... recipe = collective.buildbot:master
        ... port = 8080
        ... wport = 8082
        ... project-name = The project
        ... project-url = http://example.com
        ... url = http://example.com/buildbot
        ... slaves =
        ...     slave1 password
        ...     slave2 password
        ... """)
        
        Running the buildout gives us::
        
        >>> print system(buildout)
        Installing buildmaster...
        New python executable in /sample-buildout/parts/buildmaster/bin/python
        Installing setuptools.............done.
        Generated script '/sample-buildout/parts/buildmaster/buildbot.tac'.
        Generated config '/sample-buildout/parts/buildmaster/buildbot.cfg'.
        Generated script '/sample-buildout/bin/buildmaster'.
        
        As shown above, the buildout generated the required configuration
        files and the runner script under ``bin``. You can control build
        master process by running::
        
        $ ./bin/buildmaster [start | stop | restart]
        
        
        The Twisted .tac file that is used to launch the buildbot process::
        
        >>> cat(join('parts', 'buildmaster', 'buildbot.tac'))
        from twisted.application import service
        from buildbot.master import BuildMaster
        import os
        import sys
        import collective.buildbot
        <BLANKLINE>
        basedir = '/sample-buildout/parts/buildmaster'
        buildbot = os.path.dirname(collective.buildbot.__file__)
        <BLANKLINE>
        configfile = os.path.join(buildbot, 'master.py')
        application = service.Application('buildmaster')
        <BLANKLINE>
        master = BuildMaster(basedir, configfile)
        master.setServiceParent(application)
        <BLANKLINE>
        
        We can also see that the configuration file generated by the recipe
        reflects the options we chose in our buildout configuration::
        
        >>> cat(join('parts', 'buildmaster', 'buildbot.cfg'))
        [slaves]
        slave1 = password
        slave2 = password
        <BLANKLINE>
        [buildbot]
        projects-directory = /sample-buildout/parts/projects
        project-name = The project
        pollers-directory = /sample-buildout/parts/pollers
        url = http://example.com/buildbot
        wport = 8082
        project-url = http://example.com
        port = 8080
        allow-force = false
        <BLANKLINE>
        
        The build slave recipe
        **********************
        
        The ``collective.buildbot:slave`` recipe produces a configuration
        file that sets up a build slave process. Once the build slave is
        configured you can run it by executing the controller script under the
        buildout's bin directory. The controller script will be named after
        the section name, so if you had a ``[buildslave]`` section in your
        buildout.cfg you would get a ``bin/buildslave`` script.
        
        Since the name of the section using this recipe will also become the
        name of the build slave it is important to choose the name that
        corresponds to the buildmaster configuration.
        
        Supported options
        =================
        
        The recipe supports the following options:
        
        ``host``
        Hostname of the build master.
        
        ``port``
        Port that the build master is listening. This should match the
        ``port`` option in the section using the
        ``collective.buildbot:master`` recipe in your buildmaster
        buildout.
        
        ``password``
        Build slave password. This should match the password in the
        ``slave-names`` section in the buildmaster buildout.
        
        Example usage
        =============
        
        We'll start by creating a buildout that uses the recipe::
        
        >>> write('buildout.cfg',
        ... """
        ... [buildout]
        ... parts =
        ...	   buildslave
        ...
        ... [buildslave]
        ... recipe = collective.buildbot:slave
        ... host = localhost
        ... port = 8888
        ... password = password
        ... """)
        
        Running the buildout gives us::
        
        >>> print system(buildout)
        Installing buildslave.
        ...
        Generated script /sample-buildout/parts/buildslave/buildbot.tac.
        Generated script '/sample-buildout/bin/buildslave'.
        <BLANKLINE>
        
        As shown above, the buildout generated the required scripts. You can
        control build slave process by running::
        
        $ ./bin/buildslave [start | stop | restart]
        
        The project recipe
        ******************
        
        The ``collective.buildbot:project`` recipe is responsible for creating
        the buildbot configuration for a project which is a single testable
        component. Whether this project corresponds to a single sofware
        package or many is up to you. In most cases a project corresponds to a
        buildout which in turn may contain one or many software packages. Each
        project has a separate state and is visualized as a column in the
        waterfall display.
        
        This recipe should be used in the same buildout with
        ``collective.buildbot:master``.
        
        
        Supported options
        =================
        
        The recipe supports the following options:
        
        ``slave-names`` (mandatory)
        
        A white-space separated list of slave names that the project will be
        built on. These must correspond to the section names that use the
        ``collective.buildbot:slave`` recipe and that are consequently
        referred in the ``slaves`` option of the section using the
        ``collective.buildbot:master`` recipe.
        
        ``vcs`` (optional)
        
        The version control system used to obtain the source code for the
        project. Defaults to ``svn``. Other possible values are: ``hg``,
        ``bzr`` and ``git``.
        
        ``repository`` (mandatory)
        
        The URL to the code repository that corresponds to the selected
        version control system. For Subversion this could be something like
        ``https://svn.plone.org/svn/collective/collective.buildbot/trunk``
        or for Git something like
        ``git@github.com:dokai/hexagonit-swfheader.git``.
        
        ``branch`` (optional)
        
        The branch in the version control system that will be checked
        out. For Subversion checkouts you should provide the full URL to the
        desired branch (e.g. something that ends with ``trunk`` or
        ``branches/foo``) and leave this option empty. For Git repositories
        the default value is ``master``.
        
        ``email-notification-sender`` (optional)
        
        An email address that will be used in the From: header for the
        notification messages.
        
        ``email-notification-recipients`` (optional)
        
        A newline separated sequence of email addresses the notification
        messages will be sent to.
        
        ``build-sequence`` (optional)
        
        A newline separated sequence of shell commands executed on the build
        slave after checking out the code from the repository that will
        build the project.
        
        Defaults to::
        
        bin/python bootstrap.py
        bin/buildout
        
        which is appropriate for buildout based projects.
        
        ``test-sequence`` (optional)
        
        A newline separated sequence of shell commands executed on the build
        slave to execute the test suite. Defaults to::
        
        bin/test
        
        ``periodic-scheduler`` (optional)
        
        Sets up a periodic scheduler that schedules a build every ``n``
        minutes, where ``n`` is the given integer value.
        
        ``cron-scheduler`` (optional)
        
        Sets up a cron-like scheduler that schedules a build at a given
        time. The time is configured in a crontab manner using white space
        separated values for the following fields::
        
        [minute] [hour] [day of month] [month] [day of week]
        
        The values should be integers in the approriate range for the given
        field or ``*`` (asterisk) for all values. For example to schedule a
        build at 3:00 am every night you would use::
        
        cron-scheduler = 0 3 * * *
        
        
        Example usage
        =============
        
        We'll start by creating a buildout that uses the recipe. A full
        example would propably have other sections defining the build master
        and slaves, but here we will demonstrate only the use of the
        ``collective.buildbot:project`` recipe.
        
        >>> write('buildout.cfg',
        ... """
        ... [buildout]
        ... parts = my.package
        ...
        ... [my.package]
        ... recipe = collective.buildbot:project
        ... slave-names = slave1
        ... vcs = svn
        ... repository = http://example.com/svn/my.package/trunk
        ... """)
        
        Running the buildout gives us::
        
        >>> print system(buildout)
        Installing my.package.
        Generated config '/sample-buildout/parts/projects/my.package.cfg'.
        
        As we can see, the recipe generated the project configuration file
        under the ``projects`` directory in the parts::
        
        >>> cat(join('parts', 'projects', 'my.package.cfg'))
        [project]
        ...
        name = my.package
        repository = http://example.com/svn/my.package/trunk
        ...
        slave-names = slave1
        ...
        vcs = svn
        <BLANKLINE>
        
        The projects recipe
        *******************
        
        The ``collective.buildbot:projects`` recipe is deprecated and not
        supported anymore.
        
        
        
        The poller recipe
        ******************
        
        Supported options
        =================
        
        The recipe supports the following options:
        
        Lists the projects the buildbot deal with (one project=one column) The
        values must be a section name in the configuration file.  Then each of this
        section must contain:
        
        ``vcs``
        
        The version control system. Defaults to ``svn``. Currently only
        Subversion repositories are supported.
        
        ``repository``
        
        Full URL to the branch in the code repository containing the project
        code.
        
        ``hist-max``
        
        Number of history lines to look at (Default 100).
        
        ``user``
        
        A svn user (Default None).
        
        ``password``
        
        A valid svn password for the user (Default None).
        
        ``poll-interval``
        
        Interval in seconds to check for changes.
        
        ``svn-binary``
        
        Path to the ``svn`` binary. Defaults to ``svn`` which should work if
        you have in your ``PATH``.
        
        Example usage
        =============
        
        We can define a poller to make our buildbot aware of commits::
        
        >>> write('buildout.cfg',
        ... """
        ... [buildout]
        ... parts = svnpoller
        ...
        ... [svnpoller]
        ... recipe = collective.buildbot:poller
        ... repository = http://example.com/svn/buildout/trunk
        ... user = h4x0r
        ... password = passwd
        ... """)
        
        >>> print system(buildout)
        Installing svnpoller.
        Generated config '/sample-buildout/parts/pollers/svnpoller.cfg'.
        
        Poller generation. You can see here all the available options::
        
        >>> cat(join('parts', 'pollers', 'svnpoller.cfg'))
        [poller]
        hist-max = 100
        repository = http://example.com/svn/buildout/trunk
        vcs = svn
        user = h4x0r
        svn-binary = svn
        password = passwd
        poll-interval = 60
        <BLANKLINE>
        
        
        The pollers recipe
        *******************
        
        The ``collective.buildbot:pollers`` is similar to the
        ``collective.buildbot:poller`` recipe and the only difference is that
        it allows you to define multiple pollers in one buildout section.
        
        Supported options
        =================
        
        The options are otherwise the same except that ``repository`` is
        replaced by ``repositories``.
        
        ``repositories``
        
        A sequence of white space separated repository URLs to poll.
        
        
        
        Putting it all together
        ***********************
        
        Below we will demonstrate how to put all the pieces together to create
        a buildbot environment for your own projects. We will use two separate
        buildouts: one for the build master and one for a single build
        slave. For your own projects you may choose to use multiple build
        slaves with each running, for example, on a different architecture or
        a different python version.
        
        We'll start with the build master buildout that defines the build
        master process and all the projects that we wish to build and test. We
        also include a poller configuration that will poll the Subversion
        repository for changes the projects and execute the build when changes
        have occurred. If we were to use another version control system, such
        as Git, we would need to use a commit-hook or a time-based build
        scheduler.
        
        >>> write('buildout.cfg',
        ... """
        ... [buildout]
        ... parts =
        ...     buildmaster
        ...     svnpoller
        ...     my.project
        ...     my.buildout
        ...     another.package
        ...
        ... [buildmaster]
        ... recipe = collective.buildbot:master
        ... port = 8080
        ... wport = 8082
        ... project-name = The company buildout
        ... project-url = http://my.company.com
        ... url = http://buildbot.my.company.com
        ... allow-force = true
        ... public-html = ${buildout:directory}/buildbot_css
        ... slaves =
        ...     slave1 secretpassword
        ...
        ... [svnpoller]
        ... recipe = collective.buildbot:pollers
        ... repositories =
        ...     ${my.project:repository}
        ...     ${my.buildout:repository}
        ... user = someuser
        ... password = anothersecret
        ...
        ... [my.project]
        ... recipe = collective.buildbot:project
        ... slave-names = slave1
        ... repository = https://svn.company.com/svn/my.project/trunk
        ... email-notification-sender = buildbot@my.company.com
        ... email-notification-recipient =
        ...     my.project@my.company.com
        ...     dev@my.company.com
        ... build-sequence =
        ... test-sequence = ../../bin/python setup.py test
        ...
        ... [my.buildout]
        ... recipe = collective.buildbot:project
        ... slave-names = slave1
        ... repository = https://svn.company.com/svn/my.buildout/trunk
        ... email-notification-sender = buildbot@my.company.com
        ... email-notification-recipient = dev@my.company.com
        ... test-sequence = bin/zope-instance test -v -vv
        ...
        ... [another.package]
        ... recipe = collective.buildbot:project
        ... slave-names = slave1
        ... vcs = git
        ... repository = git://git.company.com/projects/another-package.git
        ... email-notification-sender = buildbot@my.company.com
        ... email-notification-recipient = dev@my.company.com
        ... build-sequence =
        ... test-sequence = ../../bin/python setup.py test
        ... periodic-scheduler = 60
        ... """)
        
        We've allowed forced builds which is quite handy sometimes. Since the
        default buildbot web interface is not the most aesthetic we've also
        included a directory that contains our custom css.
        
        The ``my.project`` and ``another.package`` packages are simple python
        packages so we use the setup.py script to run the test suites. Because
        these are simple packages we also clear out the ``build-sequence``
        option since there is nothing to do before running the tests. The
        ``my.buildout`` section is your typical Zope buildout and uses the
        Zope controller script, ``zope-instance`` in this particular case, to
        run the tests.
        
        Also, because the ``another.package`` project uses a Git repository,
        the SVN poller won't apply to it so we've set up a periodic scheduler
        that builds the project once in an hour. An alternative would be to
        install a post-commit hook to the Git repository that notifies the
        buildout of changes and schedules a build.
        
        The ``my.buildout`` project is a buildout based project, so we can use
        the default ``build-sequence`` which will bootstrap and run the
        buildout for us. For the zope.testing test runner we pass the
        ``--exit-with-status`` parameter so that buildbot will know whether
        the tests failed or not. The trunk may have additional svn:externals
        defined that actually pull in the code that is tested which is the
        common place.
        
        Let's run the buildout now.
        
        >>> mkdir('buildbot_css')
        >>> print system(buildout)
        Installing buildmaster.
        New python executable in /sample-buildout/parts/buildmaster/bin/python
        Installing setuptools............done.
        Generated script '/sample-buildout/parts/buildmaster/buildbot.tac'.
        Generated config '/sample-buildout/parts/buildmaster/buildbot.cfg'.
        Generated script '/sample-buildout/bin/buildmaster'.
        Installing my.project.
        Generated config '/sample-buildout/parts/projects/my.project.cfg'.
        Installing my.buildout.
        Generated config '/sample-buildout/parts/projects/my.buildout.cfg'.
        Installing svnpoller.
        Generated config '/sample-buildout/parts/pollers/poller_0.cfg'.
        Generated config '/sample-buildout/parts/pollers/poller_1.cfg'.
        Installing another.package.
        Generated config '/sample-buildout/parts/projects/another.package.cfg'.
        <BLANKLINE>
        
        As we can see we got the ``bin/buildmaster`` script to run the build
        master process and the corresponding configuration files. Our build
        master is now ready and you can start it by running::
        
        $ ./bin/buildmaster start
        
        Next, we create the buildout for the build slave. This buildout may be
        located on a different machine although having it on the same machine
        will work just as fine.
        
        >>> write('buildout.cfg',
        ... """
        ... [buildout]
        ... parts =
        ...	   buildslave
        ...
        ... [buildslave]
        ... recipe = collective.buildbot:slave
        ... host = buildbot.my.company.com
        ... port = 8080
        ... password = secretpassword
        ... """)
        
        The slave buildout is very simple since the build master is in charge
        of everything and the slave simply needs to contact the master and
        receive instructions. We confifured the address of the build master
        and the password to match the configuration in the build master
        buildout above.
        
        Running the buildout will give us the controller script::
        
        >>> print system(buildout)
        Uninstalling...
        Installing buildslave.
        New python executable in /sample-buildout/parts/buildslave/bin/python
        Installing setuptools............done.
        Generated script /sample-buildout/parts/buildslave/buildbot.tac.
        Generated script '/sample-buildout/bin/buildslave'.
        <BLANKLINE>
        
        The build slave can be started by running::
        
        $ ./bin/buildslave start
        
        Once you have both the build master and slave running the poller
        should react to commits to the SVN repositories and run the builds
        after each change. You can view the buildbot status pages at the
        configured address, http://buildbot.my.company.com:8082/ in this
        case. You can use the web interface to force a build which can be
        useful to verify that the buildbot and projects are configured
        correctly.
        
        It is now easy to add new projects or build slaves by modifying the
        buildout configurations and rerunning the buildouts.
        
        Contributors
        ************
        
        Project initiated at Ingeniweb (http://ingeniweb.com)
        
        Initial Authors:
        - Gael Pasgrimaud
        - Tarek Ziade
        
        Moved to the collective during the Plone Paris Sprint 2008.
        
        Contributors:
        - Gael Pasgrimaud [gawel]
        - Tarek Ziade [tarek]
        - Kai Lautaportti [dokai]
        - Jean-Francois Roche
        - Mustapha Benali [mustapha]
        
        Download
        ********
        
Keywords: buildout buildbot
Platform: UNKNOWN
Classifier: Framework :: Buildout
Classifier: Intended Audience :: Developers
Classifier: Topic :: Software Development :: Build Tools
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: License :: OSI Approved :: Zope Public License
